ONOS org.onosproject.onos-apps-segmentrouting-app crash on aarch64

135 views
Skip to first unread message

cipria...@gmail.com

unread,
Sep 19, 2019, 5:48:32 AM9/19/19
to SEBA Developers, z...@opennetworking.org, t...@opennetworking.org, ji...@cachengo.com
Hello,

First of all I apologize if this is not the appropriate forum, I assume there should be a mailing list for ONOS, but I can't figure out which is it.

In the context of Akrino IEC, I've been investigating a problem on a SEBA-in-a-Box installation, at least sort of, because I'm deploying SEBA using the cord-platform -> seba -> att-workflow set of charts and then install PONSim. All of these use manually built Docker images that work on aarch64!

Right now I can do the authentication step, but the DHCP fails, although I can see the DHCP replies coming from the BNG in the mininet POD. Which then revealed that ONOS is not catching them.
There are a lot of details in an Akraino Jira card:

Right now the most obvious problem is that Xconnet Manager crashes during what looks to be the operation of adding the corresponding flows in ONOS. Not all the flows are lost, by comparing against a working setup on x86 there were suppose to be 6 new flows added, but only 4 got there.
The problem is reproducible with the same outcome every time. Here is the relevant bactrace in the ONOS log:

2019-09-13 15:09:55,807 | INFO  | AAA-radius-0     | AaaManager                       | 185 - org.opencord.aaa - 1.8.0 | Auth event APPROVED for of:0000aabbccddeeff/128
2019-09-13 15:10:23,278 | INFO  | qtp300471503-38  | Olt                              | 187 - org.opencord.olt-app - 2.1.0 | Programming vlans for subscriber: [id:PSMO12345678,cTag:111,sTag:222,nasPortId:,uplinkPort:-1,slot:-1,hardwareIdentifier:null,ipaddress:null,nasId:null,circuitId:,remoteId:]
2019-09-13 15:10:23,287 | INFO  | ispatch-default0 | AccessDeviceKafkaIntegration     | 189 - org.opencord.kafka - 1.0.0 | Got AccessDeviceEvent: SUBSCRIBER_REGISTERED
2019-09-13 15:10:23,350 | INFO  | ce-operations-29 | Olt                              | 187 - org.opencord.olt-app - 2.1.0 | DHCP v4 filter for device of:0000aabbccddeeff on port 128 installed.
2019-09-13 15:10:23,350 | WARN  | tive-installer-3 | OltPipeline                      | 161 - org.onosproject.onos-drivers-default - 1.13.5 |
Only the following are Supported in OLT for filter ->
ETH TYPE : EAPOL, LLDP and IPV4
IPV4 TYPE: IGMP and UDP (for DHCP)
2019-09-13 15:10:23,351 | WARN  | tive-installer-3 | InOrderFlowObjectiveManager      | 130 - org.onosproject.onos-core-net - 1.13.5 | Flow objective onError DefaultFilteringObjective{id=1084928967, type=PERMIT, op=ADD, priority=10000, key=IN_PORT:128, conditions=[ETH_TYPE:ipv6, IP_PROTO:17, UDP_SRC:547, UDP_DST:546], meta=DefaultTrafficTreatment{immediate=[OUTPUT:CONTROLLER], deferred=[], transition=None, meter=[], cleared=false, StatTrigger=null, metadata=null}, appId=DefaultApplicationId{id=176, name=org.opencord.olt}, permanent=true, timeout=0}. Reason = UNSUPPORTED
2019-09-13 15:10:23,351 | INFO  | tive-installer-3 | Olt                              | 187 - org.opencord.olt-app - 2.1.0 | DHCP v6 filter for device of:0000aabbccddeeff on port 128 failed installation because UNSUPPORTED
2019-09-13 15:10:25,097 | INFO  | qtp300471503-38  | XconnectManager                  | 177 - org.onosproject.onos-apps-segmentrouting-app - 1.13.5 | Adding or updating xconnect. deviceId=of:0000000000000001, vlanId=222, ports=[1, 2]
2019-09-13 15:10:40,118 | ERROR | nt-partition-1-0 | ThreadPoolContext                | 90 - io.atomix - 2.0.23 | An uncaught exception occurred
java.lang.IllegalStateException: org.onosproject.store.service.ConsistentMapException$Timeout: onos-sr-xconnect-next
        at org.onosproject.store.primitives.impl.MeteredAsyncConsistentMap$InternalMeteredMapEventListener.event(MeteredAsyncConsistentMap.java:310)
        at org.onosproject.store.primitives.impl.CachingAsyncConsistentMap.lambda$null$1(CachingAsyncConsistentMap.java:93)
        at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:399)[95:com.google.guava:22.0.0]
        at org.onosproject.store.primitives.impl.CachingAsyncConsistentMap.lambda$null$2(CachingAsyncConsistentMap.java:93)
        at java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1597)[:1.8.0_201]
        at org.onosproject.store.primitives.impl.CachingAsyncConsistentMap.lambda$new$3(CachingAsyncConsistentMap.java:93)
        at org.onosproject.store.primitives.impl.TranscodingAsyncConsistentMap$InternalBackingMapEventListener.event(TranscodingAsyncConsistentMap.java:366)
        at org.onosproject.store.primitives.impl.TranscodingAsyncConsistentMap$InternalBackingMapEventListener.event(TranscodingAsyncConsistentMap.java:366)
        at org.onosproject.store.primitives.resources.impl.AtomixConsistentMap.lambda$null$1(AtomixConsistentMap.java:128)[133:org.onosproject.onos-core-primitives:1.13.5]
        at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:399)[95:com.google.guava:22.0.0]
        at org.onosproject.store.primitives.resources.impl.AtomixConsistentMap.lambda$null$2(AtomixConsistentMap.java:128)[133:org.onosproject.onos-core-primitives:1.13.5]
        at java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1597)[:1.8.0_201]
        at org.onosproject.store.primitives.resources.impl.AtomixConsistentMap.lambda$handleEvent$3(AtomixConsistentMap.java:128)[133:org.onosproject.onos-core-primitives:1.13.5]
        at java.util.ArrayList.forEach(ArrayList.java:1257)[:1.8.0_201]
        at org.onosproject.store.primitives.resources.impl.AtomixConsistentMap.handleEvent(AtomixConsistentMap.java:127)[133:org.onosproject.onos-core-primitives:1.13.5]
        at io.atomix.protocols.raft.proxy.impl.DelegatingRaftProxy.lambda$addEventListener$4(DelegatingRaftProxy.java:122)
        at io.atomix.protocols.raft.proxy.impl.BlockingAwareRaftProxyClient.lambda$null$2(BlockingAwareRaftProxyClient.java:67)
        at io.atomix.utils.concurrent.ThreadPoolContext.lambda$new$0(ThreadPoolContext.java:81)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_201]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_201]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)[:1.8.0_201]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)[:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748)[:1.8.0_201]
Caused by: org.onosproject.store.service.ConsistentMapException$Timeout: onos-sr-xconnect-next
        at org.onosproject.store.primitives.DefaultConsistentMap.complete(DefaultConsistentMap.java:258)
        at org.onosproject.store.primitives.DefaultConsistentMap.containsKey(DefaultConsistentMap.java:77)
        at org.onosproject.segmentrouting.xconnect.impl.XconnectManager.populateNext(XconnectManager.java:485)
        at org.onosproject.segmentrouting.xconnect.impl.XconnectManager.populateXConnect(XconnectManager.java:455)
        at org.onosproject.segmentrouting.xconnect.impl.XconnectManager.access$300(XconnectManager.java:98)
        at org.onosproject.segmentrouting.xconnect.impl.XconnectManager$XconnectMapListener.event(XconnectManager.java:297)
        at org.onosproject.store.primitives.impl.MeteredAsyncConsistentMap$InternalMeteredMapEventListener.event(MeteredAsyncConsistentMap.java:306)
        ... 24 more


At this point I can't match the trace against Xconnect version 1.13.5, I have no idea how the app is built and installed in the Docker image.

What information I received so far, the aarch64 Docker image has been built using this Dockerfile:

The cord-platform chart and other related charts can be found here:

PONSim charts can be found for now at:

Any help is greatly appreciated as this points to a serious problem on aarch64 which needs to be fixed.

Thanks and regards,
Ciprian

Charles Chan

unread,
Sep 19, 2019, 1:15:43 PM9/19/19
to cipria...@gmail.com, SEBA Developers, Zack Williams, Matteo Scandolo, ji...@cachengo.com, trellis-dev
The exception is unlikely related to different CPU architecture. It is mainly about the distributed store time out when XConnectManager tries to write to the store.

Have you tried ONOS 1.13.9? This patch may solve the problem, but it is not part of 1.13.5.

Thanks,
Charles Chan, Ph.D.
Member of Technical Staff, Open Networking Foundation


--
You received this message because you are subscribed to the Google Groups "SEBA Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seba-dev+u...@opennetworking.org.
To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/seba-dev/6e617200-c56f-450b-abfc-c8c1a904b6fc%40opennetworking.org.

Tina Tsou

unread,
Sep 19, 2019, 5:07:04 PM9/19/19
to Charles Chan, lxi...@vmware.com, Matteo Scandolo, SEBA Developers, Zack Williams, cipria...@gmail.com, ji...@cachengo.com, trellis-dev
+ Xinhui for xConnect.

--
Thank you, Tina

cipria...@gmail.com

unread,
Sep 20, 2019, 7:03:20 AM9/20/19
to SEBA Developers, cipria...@gmail.com, z...@opennetworking.org, t...@opennetworking.org, ji...@cachengo.com, trell...@opennetworking.org
Hi Charles,

Thank you for the suggestion. I will try to test with version 1.13.9 as indicated.
I assume the segmentrouting app is built together with ONOS, so I will need to take the newer ONOS altogether. I will need some time to built a new image for aarch64 and get back to you as soon as possible.

Thank and regards,
/Ciprian
To unsubscribe from this group and stop receiving emails from it, send an email to seba...@opennetworking.org.

Zack Williams

unread,
Sep 30, 2019, 12:17:35 PM9/30/19
to cipria...@gmail.com, SEBA Developers, Matteo Scandolo, Jimmy Lafontaine Rivera, trell...@opennetworking.org, Xinhui Li
Hi Ciprian,

Did upgrading to 1.13.9 help resolve this issue?

Thanks,
Zack
---
Zack Williams
z...@opennetworking.org

cipria...@gmail.com

unread,
Sep 30, 2019, 6:44:46 PM9/30/19
to SEBA Developers, cipria...@gmail.com, t...@opennetworking.org, ji...@cachengo.com, trell...@opennetworking.org, lxi...@vmware.com
Hello Zack,

Unfortunately the timing was bad for me to start this thread in the first place, I went into a 3 weeks vacation starting last week.
Fortunately this work has been taken over by other colleagues and I think they made a preliminary build which might be tested this week.
We will update as soon as possible, for now things are going a bit slow.

Thanks and regards,
/Ciprian

cipria...@gmail.com

unread,
Oct 14, 2019, 8:32:41 AM10/14/19
to SEBA Developers, cipria...@gmail.com, t...@opennetworking.org, ji...@cachengo.com, trell...@opennetworking.org, lxi...@vmware.com
Hello guys,

I'm back from vacation now. There was no update on this thread since two weeks ago, but my colleagues have managed to build an ONOS 1.13.9 Docker image which unfortunately yields the same result.
I need to look into the details, but I think the flows are again not programmed properly, so at this point we are still kind of blocked on identifying this issue.

BR,
Ciprian

Tina Tsou

unread,
Oct 15, 2019, 12:04:06 PM10/15/19
to cipria...@gmail.com, SEBA Developers, ji...@cachengo.com, lxi...@vmware.com, t...@opennetworking.org, trell...@opennetworking.org
Dear all,

Is there SEBA meeting today?

I’m waiting at the GoToMeeting bridge.

To unsubscribe from this group and stop receiving emails from it, send an email to seba-dev+u...@opennetworking.org.
To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/seba-dev/ec95904f-9f60-4b68-95bb-86b0e409c1e3%40opennetworking.org.
--
Thank you, Tina

Saurav Das

unread,
Oct 15, 2019, 12:39:13 PM10/15/19
to Tina Tsou, cipria...@gmail.com, SEBA Developers, ji...@cachengo.com, Xinhui Li, Matteo Scandolo, trell...@opennetworking.org
Hi Tina,

SEBA meetings have been temporarily suspended as the community focuses on VOLTHA and meets frequently in the VOLTHA calls or the various brigade calls.
We will resume the SEBA calls at a future date, with advance notice.
Sorry for the confusion

Thanks
Saurav



Tina Tsou

unread,
Oct 15, 2019, 3:00:48 PM10/15/19
to Saurav Das, Matteo Scandolo, SEBA Developers, Xinhui Li, cipria...@gmail.com, ji...@cachengo.com, trell...@opennetworking.org
Dear Saurav,

Thanks for the update.

We only need someone to answer Ciprian’s questions from Akraino SEBA blueprint’s upstream requests.

Would you kindly answer his question he posted and reposted at seba mailing list? It has been a month.
--
Thank you, Tina

Saurav Das

unread,
Oct 15, 2019, 3:30:43 PM10/15/19
to Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, cipria...@gmail.com, ji...@cachengo.com, trell...@opennetworking.org
Sure no problem. First question -- Is this a SEBA 1.0 pod which used ONOS 1.13.5 and related apps, or is this a SEBA 2.0 pod which used ONOS 1.13.9 and related apps.
You cannot use 1.13.9 and still use the older apps for 1.13.5.

Regarding your problem with DHCP
do you have this flow?

Thanks
Saurav


Tina Tsou

unread,
Oct 16, 2019, 9:50:40 AM10/16/19
to Saurav Das, Matteo Scandolo, SEBA Developers, Xinhui Li, cipria...@gmail.com, ji...@cachengo.com, trell...@opennetworking.org
Dear Ciprian,

Would you please answer it?
--
Thank you, Tina

Ciprian Barbu

unread,
Oct 16, 2019, 12:11:44 PM10/16/19
to Tina Tsou, Saurav Das, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trell...@opennetworking.org
Hi,

Sorry for the late reply, I've used my personal email for this thread, my bad.

This is SEBA 1.0 with the related apps. I too realized that using ONOS 1.13.9 might not be a good idea, after analyzing the flows and logs and noticing things are not quite in order.

The flow you pointed is not the issue, I did some extensive work capturing logs and flows and comparing them to a working (x86) setup. From this work, I ended up with the following set of flows and flow groups that are missing on the aggregation switch (Openvswitch in this case). In fact, I'm showing the output from ovs-ofctl:

Missing flows:
=====================================
 cookie=0xac00004df3cb3e, duration=593.794s, table=10, n_packets=0, n_bytes=0, send_flow_rem in_port=1,dl_vlan=222 actions=goto_table:20
 cookie=0xac00003d987c88, duration=593.794s, table=10, n_packets=0, n_bytes=0, send_flow_rem in_port=2,dl_vlan=222 actions=goto_table:20
 cookie=0xac000074fa5059, duration=593.042s, table=50, n_packets=0, n_bytes=0, send_flow_rem priority=1000,dl_vlan=222 actions=write_actions(group:1088290816),goto_table:60
 cookie=0xac000062d0f17c, duration=593.779s, table=60, n_packets=0, n_bytes=0, send_flow_rem priority=60000 actions=drop
 cookie=0xb20000420c9478, duration=593.794s, table=63, n_packets=0, n_bytes=0, send_flow_rem priority=40000,in_port=2,dl_vlan=222 actions=CONTROLLER:65535
 cookie=0xb2000001f9e66b, duration=593.794s, table=63, n_packets=0, n_bytes=0, send_flow_rem priority=40000,in_port=1,dl_vlan=222 actions=CONTROLLER:65535

Missing groups:
=====================================
 group_id=14548993,type=indirect,bucket=actions=output:1
 group_id=14548994,type=indirect,bucket=actions=output:2
 group_id=1088290816,type=all,bucket=actions=group:14548993,bucket=actions=group:14548994

So as you can see I'm missing flows that handle the double tagged packets sent from the BNG (simulated by mininet app), to the aggregation switch.

But I think I might have another option, which is rebuilding ONOS 1.13.5 together with the aforementioned patch (by Charles Chan. Hopefully there won't be other dependencies.

I will update you as soon as I have news.
BR,
/Ciprian

Saurav Das

unread,
Oct 16, 2019, 1:01:01 PM10/16/19
to Ciprian Barbu, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trell...@opennetworking.org
But these missing flows and groups have to do with the vlan-xconnect. 
They should not have any impact on dhcp.
You said dhcp was not working -- is that working now?

Ciprian Barbu

unread,
Oct 16, 2019, 1:11:59 PM10/16/19
to Saurav Das, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trell...@opennetworking.org
They do have an impact, because the DHCP replies from the simulated BNG are tagged. They never reach the agg switch. But the requests from the RG pod do reach the BNG.

Saurav Das

unread,
Oct 16, 2019, 1:32:39 PM10/16/19
to Ciprian Barbu, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trell...@opennetworking.org
The vlan-tagged replies from the DHCP server for DHCP packets should not go through the AGG switch dataplane. 
They should be trapped to the controller from the AGG switch if there is a flow like this in table 60

requires this flow
ADDED, bytes=0, packets=0, table=60, priority=61000, selector=[IN_PORT:32, ETH_TYPE:ipv4, IP_PROTO:17, UDP_SRC:67, UDP_DST:68], treatment=[immediate=[OUTPUT:CONTROLLER], clearDeferred]

It is possible the SiaB in SEBA 1.0 did not work like this, and instead depended on a vlan-xconnect (which is not working for you)
But my question is - do you see such a flow?

Thanks
Saurav




Ciprian Barbu

unread,
Oct 16, 2019, 1:55:14 PM10/16/19
to Saurav Das, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trell...@opennetworking.org
I need to double check, but I'm quite certain I have this flow, I'm no longer at the office right now.

You're right that the packets need to be trapped by the switch and sent to the controller, I didn't express myself correctly earlier.

I tried to use ovs-appctl to understand how these packets should travel through the flow tables, but I couldn't reach a conclusion.

But I can tell you for sure that the DHCP requests arrive to the dnsmasq process, double tagged with tags 222 and 111 (these are hardcoded by the ponsim charts). Then the dnsmasq replies with DHCP offer, and the packets are tagged back and sent on the same interfaces where the requests originated from.

At this point, ovs receives them, but they are never sent to the controller. They also do not reach the dhcpl2relay app in ONOS. The DHCP requests however can be seen in the dhcpl2relay trace in ONOS. So you see, the problem must be the missing flows and flow groups, I'm just confused how it is supposed to work.

Charles Chan

unread,
Oct 16, 2019, 2:54:04 PM10/16/19
to Ciprian Barbu, Saurav Das, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trellis-dev
 Below is what I get in latest SiaB.

onos> flows -s any of:0000000000000001 60
deviceId=of:0000000000000001, flowRuleCount=13
    ADDED, bytes=1072, packets=3, table=60, priority=61000, selector=[IN_PORT:1, ETH_TYPE:ipv4, IP_PROTO:17, UDP_SRC:67], treatment=[clearDeferred, transition=TABLE:63]
    ADDED, bytes=850, packets=12, table=60, priority=60000, selector=[VLAN_VID:222], treatment=[immediate=[NOACTION]]

    ADDED, bytes=0, packets=0, table=60, priority=40000, selector=[ETH_TYPE:eapol], treatment=[clearDeferred, transition=TABLE:63]
    ADDED, bytes=0, packets=0, table=60, priority=40000, selector=[ETH_TYPE:lldp], treatment=[clearDeferred, transition=TABLE:63]
    ADDED, bytes=9460, packets=110, table=60, priority=40000, selector=[ETH_TYPE:bddp], treatment=[clearDeferred, transition=TABLE:63]
    ADDED, bytes=0, packets=0, table=60, priority=40000, selector=[ETH_TYPE:ipv4, IPV4_DST:192.168.0.201/32], treatment=[clearDeferred, transition=TABLE:63]
    ADDED, bytes=0, packets=0, table=60, priority=40000, selector=[ETH_TYPE:ipv6, IPV6_DST:fe80::200:2ff:fe01:601/128], treatment=[clearDeferred, transition=TABLE:63]
    ADDED, bytes=0, packets=0, table=60, priority=30000, selector=[ETH_TYPE:arp], treatment=[transition=TABLE:63]
    ADDED, bytes=0, packets=0, table=60, priority=30000, selector=[ETH_TYPE:ipv6, IP_PROTO:58, ICMPV6_TYPE:134], treatment=[transition=TABLE:63]
    ADDED, bytes=0, packets=0, table=60, priority=30000, selector=[ETH_TYPE:ipv6, IP_PROTO:58, ICMPV6_TYPE:135], treatment=[transition=TABLE:63]
    ADDED, bytes=0, packets=0, table=60, priority=30000, selector=[ETH_TYPE:ipv6, IP_PROTO:58, ICMPV6_TYPE:136], treatment=[transition=TABLE:63]
    ADDED, bytes=1542, packets=21, table=60, priority=30000, selector=[ETH_TYPE:ipv6, IP_PROTO:58, ICMPV6_TYPE:133], treatment=[transition=TABLE:63]
    ADDED, bytes=4294, packets=62, table=60, priority=0, selector=[], treatment=[immediate=[NOACTION]]

I can reproduce the same issue if I manually remove xconnect. DHCP no longer works in that case.
I think the problem is in table 63, the packet-in table we implement in OVS to preserve original VLAN of the packet.

onos> flows -s any of:0000000000000001 63
deviceId=of:0000000000000001, flowRuleCount=4
    ADDED, bytes=19436, packets=226, table=63, priority=65535, selector=[ETH_TYPE:bddp], treatment=[immediate=[OUTPUT:CONTROLLER]]
    ADDED, bytes=0, packets=0, table=63, priority=65535, selector=[ETH_TYPE:lldp], treatment=[immediate=[OUTPUT:CONTROLLER]]
    ADDED, bytes=0, packets=0, table=63, priority=40000, selector=[IN_PORT:2, VLAN_VID:4090], treatment=[immediate=[OUTPUT:CONTROLLER]]
    ADDED, bytes=0, packets=0, table=63, priority=40000, selector=[IN_PORT:1, VLAN_VID:4090], treatment=[immediate=[OUTPUT:CONTROLLER]]

There should be a flow matching on VLAN=222, IN_PORT=1 and send to CONTROLLER
I guess the problem didn't surface with xconnect installed since the packet will go through ASG data plane and being packet_in at OLT.

I will follow up with a proposed solution soon.

Thanks,
Charles Chan, Ph.D.
Member of Technical Staff, Open Networking Foundation

Charles Chan

unread,
Oct 16, 2019, 2:57:48 PM10/16/19
to Ciprian Barbu, Saurav Das, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trellis-dev
To clarify, there will be such table 63 flow when xconnect is installed.
In that case, the DHCP packet will first hit table 60

ADDED, bytes=1072, packets=3, table=60, priority=61000, selector=[IN_PORT:1, ETH_TYPE:ipv4, IP_PROTO:17, UDP_SRC:67], treatment=[clearDeferred, transition=TABLE:63]
and then table 63 
ADDED, bytes=0, packets=0, table=63, priority=40000, selector=[IN_PORT:1, VLAN_VID:222], treatment=[immediate=[OUTPUT:CONTROLLER]]


Saurav Das

unread,
Oct 16, 2019, 3:16:46 PM10/16/19
to Charles Chan, Ciprian Barbu, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trellis-dev
Thanks Charles, in the meantime, sounds like Ciprian was right - his problem will be solved if the xconnect works, the AGG switch forwards the dhcp packets to the OLT, and the OLT traps the packets to the controller.
The xconnect needs to work anyway for all other dataplane traffic to work. 
But the problem he is seeing with the xconnect is not one we have encountered with SiaB before - nevertheless lets go with the approach Ciprian is trying now to rebuild 1.13.5 with your patch and try again

Charles Chan

unread,
Oct 17, 2019, 11:17:03 PM10/17/19
to Saurav Das, Pier Luigi Ventre, Ciprian Barbu, Tina Tsou, Matteo Scandolo, SEBA Developers, Xinhui Li, ji...@cachengo.com, trellis-dev


Thanks,
Charles Chan, Ph.D.

Member of Technical Staff, Open Networking Foundation

cipria...@gmail.com

unread,
Oct 18, 2019, 12:15:23 PM10/18/19
to SEBA Developers, saura...@opennetworking.org, pi...@opennetworking.org, cipria...@gmail.com, tina...@gmail.com, t...@opennetworking.org, lxi...@vmware.com, ji...@cachengo.com, trell...@opennetworking.org
Hi Charles,

I finally managed to build ONOS 1.13.5 with the patch you first indicated. I looked for dependencies with git-cherry, I noticed the change was actually cherry-picking another commit, but in the end I did git cherry-pick 6c48dba766f0e79a65098b94bc87f5f4d29dc3ff.
As I expected, this indeed did the trick for me, now I have all the flows for the ovs switch and DHCP step completed successfully. I also pinged the BNG, which in SiaB with PONSim has a fixed value of 172.18.0.10. It works fine.
I did notice a weird behavior, sometimes I would get a bunch of missing flows from table 0 (handling lldp, eapol bddp and ipv6), and also weird some of them seemed to appear in table 60. But this doesn't seem to be related to our issue and I still can't tell when it happens.

So I think you were right on point indicating this fix to me.
About your last change, I assume it applies to latest (master) ONOS, I don't expect it will land in SEBA 1.0 as well, but if it does, let me know, so I will build another ONOS image for aarch64.

I think we can consider this fixed, thanks for all the help!

BR,
/Ciprian


On Friday, October 18, 2019 at 6:17:03 AM UTC+3, Charles Chan wrote:
Here's my proposed fix: https://gerrit.onosproject.org/#/c/22723/

Thanks,
Charles Chan, Ph.D.

Member of Technical Staff, Open Networking Foundation


--
Thank you, Tina

--
You received this message because you are subscribed to the Google Groups "SEBA Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seba...@opennetworking.org.

--
You received this message because you are subscribed to the Google Groups "SEBA Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seba...@opennetworking.org.

Saurav Das

unread,
Oct 18, 2019, 1:58:32 PM10/18/19
to Ciprian Barbu, SEBA Developers, Pier Luigi Ventre, Tina Tsou, Matteo Scandolo, Xinhui Li, ji...@cachengo.com, trellis-dev
Nice work Ciprian. Thanks Charles

Charles Chan

unread,
Oct 18, 2019, 3:04:24 PM10/18/19
to Ciprian Barbu, SEBA Developers, Saurav Das, Pier Luigi Ventre, Tina Tsou, Matteo Scandolo, Xinhui Li, ji...@cachengo.com, trellis-dev
Ciprian,

Glad that you figure it out.

The second patch (https://gerrit.onosproject.org/#/c/22723/) fixes the issue that packet-in doesn't happen correctly on OVS under certain circumstances.
In real OF-DPA hardware, the DHCP flow in ACL [1] should send the packet to controller, even when the cross connect [2] is not setup. That didn't happen in OVS without the patch.
I just discussed with Saurav. We both agree that this is a critical issue and we decided to back port this to all ONOS LTS versions as well as 1.13.
Saurav mentioned that there is currently no plan to cut another release for SEBA 1.0.
Having said that, I would still encourage you to cherry-pick the second patch and try it out.

Please find the rest of my reply inline.

[1] ADDED, bytes=1072, packets=3, table=60, priority=61000, selector=[IN_PORT:1, ETH_TYPE:ipv4, IP_PROTO:17, UDP_SRC:67], treatment=[clearDeferred, transition=TABLE:63]
[2] More accurately, the filtering objective generated by cross connect


Thanks,
Charles Chan, Ph.D.

Member of Technical Staff, Open Networking Foundation


On Fri, Oct 18, 2019 at 9:15 AM <cipria...@gmail.com> wrote:
Hi Charles,

I finally managed to build ONOS 1.13.5 with the patch you first indicated. I looked for dependencies with git-cherry, I noticed the change was actually cherry-picking another commit, but in the end I did git cherry-pick 6c48dba766f0e79a65098b94bc87f5f4d29dc3ff.
As I expected, this indeed did the trick for me, now I have all the flows for the ovs switch and DHCP step completed successfully. I also pinged the BNG, which in SiaB with PONSim has a fixed value of 172.18.0.10. It works fine.
I did notice a weird behavior, sometimes I would get a bunch of missing flows from table 0 (handling lldp, eapol bddp and ipv6), and also weird some of them seemed to appear in table 60. But this doesn't seem to be related to our issue and I still can't tell when it happens.

[Charles] Table 0 flow typically indicates that the network config is not correctly loaded when the device (OVS) first connecting to ONOS.
This can be resolved by disconnecting and reconnecting OVS to ONOS.
 
To unsubscribe from this group and stop receiving emails from it, send an email to seba-dev+u...@opennetworking.org.
To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/seba-dev/98249a8a-bc5d-42eb-ae4e-eb8a0c44b59d%40opennetworking.org.

cipria...@gmail.com

unread,
Nov 1, 2019, 10:38:27 AM11/1/19
to SEBA Developers, cipria...@gmail.com, saura...@opennetworking.org, tina...@gmail.com, t...@opennetworking.org, lxi...@vmware.com, ji...@cachengo.com, trell...@opennetworking.org
Hi Charles,

Just FYI, since I'm not a contributor on opennetworking, I backported your second patch to 1.13.5 and verified it on my setup, it's ok, no problems encountered.
I cherry-picked this to our iecedge/onos repo here:

BR,
/Ciprian

Charles Chan

unread,
Nov 1, 2019, 2:15:51 PM11/1/19
to cipria...@gmail.com, SEBA Developers, ji...@cachengo.com, lxi...@vmware.com, saura...@opennetworking.org, t...@opennetworking.org, tina...@gmail.com, trell...@opennetworking.org
Hi Ciprian,

Thanks for testing it. I will merge this shortly.

Charles

To unsubscribe from this group and stop receiving emails from it, send an email to seba-dev+u...@opennetworking.org.
To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/seba-dev/c5943d00-3806-4dc0-8385-a0d522b2ab42%40opennetworking.org.
--
Sent from mobile. Please excuse briefness and typo.
Message has been deleted

Dutafilm21 Official

unread,
Sep 23, 2021, 9:56:54 AM9/23/21
to SEBA Developers, cipria...@gmail.com
DUTAFILM21 Nonton Streaming Layarkaca21 INDOXX1 Ganool Dunia21 Lk21 Dunia21 Bioskop Cinema 21 Filmapik Pusatfilm21 Bioskopkeren Savefilm21 tempat nonton dan download film subtitle indonesia gratis terlengkap, kami juga menyediakan layanan streaming movie online seperti LK21 INDOXXI. Selain itu kami juga menyediakan tempat untuk streaming drama series gratis update tercepat dan kualitas beragam mulai dari 360p, 480p, 540p, 720p, sampai 1080p. Kebanyakan film movie dan drama series yang ada disini kualitas HD dan Bluray. Seri yang ada di SOHIB21 ini sangat lengkap, ada drama korea atau drakor, drama china atau drachin, ada juga west series atau seri barat, dan kami juga menyediakan series indonesia gratis. Berbagai film movie yang ada di SOHIB21 ini bersumber dari banyak sekali website seperti LK21, INDOXXI, Layarkaca21, GudangMovies21, JuraganFilm, Cinema Indo, Bioskop 21, Bioskop Keren, Ganool, FilmApik, dan masih banyak lainnya. Sedangkan drama series, drakor, drama china, drama indo yang ada di SOHIB21 ini bersumber dari KorDramas, DrakorIndo, DramaKoreaIndo, RatuDrama, Drakorstation, GM21, Fmoviez, FMZM, BioskopKeren, Smallencode, dan masih banyak lainnya.
Reply all
Reply to author
Forward
0 new messages