Unknown format for p4_device_config error

293 views

Skip to first unread message

Prashanth Iyengar

unread,

Dec 17, 2021, 7:18:58 PM12/17/21

to ONOS Developers

Hello,

I am trying to learn about ONOS by pushing a P4 program to a Stratum device running on a physical Barefoot Tofino switch.

The ONOS instance is running on a local machine - version 2.6.0

Stratum build - stratum-bfrt-21.10-9.4.0-amd64.deb

I attempted to write a very basic ONOS application that loads a sample P4 program - simple_l3_arping.p4. Used the build.sh script provided as part for the fabric-tofino project as a reference to build the P4 program.

The application was registered successfully.

ONOS log snippet

My netcfg.json looks like below:

{
"devices": {
"device:stratum-inventec-d10056-1": {
"basic": {
"managementAddress": "grpc://10.4.16.67:9559?device_id=1",
"driver": "stratum-tofino",
"pipeconf": "org.onosproject.foo.simple_l3_arping.tofino.montara_sde_9_4_0"
}
}
}
}

At this point, an attempt to push netcfg/pipeconf to ONOS fails - Unknown format for p4_device_config·

ONOS log snippet

2021-12-17T15:15:22,170 | INFO | onos-gdp-0 | P4RuntimeControllerImpl | 230 - org.onosproject.onos-protocols-grpc-ctl - 2.6.0 | Creating P4Runtime client for device:stratum-inventec-d10056-1...
2021-12-17T15:15:22,176 | INFO | onos-gdp-0 | GnmiControllerImpl | 230 - org.onosproject.onos-protocols-grpc-ctl - 2.6.0 | Creating gNMI client for device:stratum-inventec-d10056-1...
2021-12-17T15:15:22,220 | INFO | onos-gdp-0 | GnoiControllerImpl | 230 - org.onosproject.onos-protocols-grpc-ctl - 2.6.0 | Creating gNOI client for device:stratum-inventec-d10056-1...
2021-12-17T15:15:22,228 | INFO | onos-gdp-0 | DeviceManager | 193 - org.onosproject.onos-core-net - 2.6.0 | Local role is MASTER for device:stratum-inventec-d10056-1
2021-12-17T15:15:22,233 | WARN | onos-event-dispatch-default0 | ModelCache | 201 - org.onosproject._onos-gui2-base-jar - 2.6.0 | DeviceID device:stratum-inventec-d10056-1 not found as a UiDevice
2021-12-17T15:15:22,236 | WARN | onos-event-dispatch-default0 | ModelCache | 201 - org.onosproject._onos-gui2-base-jar - 2.6.0 | DeviceID device:stratum-inventec-d10056-1 not found as a UiDevice
2021-12-17T15:15:22,239 | INFO | onos-gdp-0 | GeneralDeviceProvider | 235 - org.onosproject.onos-providers-general-device - 2.6.0 | Notifying role MASTER (preference 0) for term 1 to device:stratum-inventec-d10056-1
2021-12-17T15:15:22,240 | WARN | onos-device-manager-background | DeviceManager | 193 - org.onosproject.onos-core-net - 2.6.0 | Node was instructed to be MASTER role for device:stratum-inventec-d10056-1, but this node cannot reach the device. Relinquishing role.··
2021-12-17T15:15:22,242 | WARN | onos-event-dispatch-default0 | ModelCache | 201 - org.onosproject._onos-gui2-base-jar - 2.6.0 | DeviceID device:stratum-inventec-d10056-1 not found as a UiDevice
2021-12-17T15:15:22,242 | WARN | onos-event-dispatch-default0 | ModelCache | 201 - org.onosproject._onos-gui2-base-jar - 2.6.0 | DeviceID device:stratum-inventec-d10056-1 not found as a UiDevice
2021-12-17T15:15:22,243 | WARN | onos-event-dispatch-default0 | ModelCache | 201 - org.onosproject._onos-gui2-base-jar - 2.6.0 | DeviceID device:stratum-inventec-d10056-1 not found as a UiDevice
2021-12-17T15:15:22,243 | INFO | onos-gdp-0 | StreamClientImpl | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.6.0 | Setting mastership on device:stratum-inventec-d10056-1... master=true, newElectionId=20, masterElectionId=null, sessionOpen=false
2021-12-17T15:15:22,292 | INFO | onos-topo-build-1 | TopologyManager | 193 - org.onosproject.onos-core-net - 2.6.0 | Topology DefaultTopology{time=106851400245026, creationTime=1639782922291, computeCost=88471, clusters=0, devices=0, links=0} changed
2021-12-17T15:15:22,296 | INFO | onos-gdp-0 | DeviceManager | 193 - org.onosproject.onos-core-net - 2.6.0 | Device device:stratum-inventec-d10056-1 registered
2021-12-17T15:15:22,333 | INFO | onos-device-manager-background | GeneralDeviceProvider | 235 - org.onosproject.onos-providers-general-device - 2.6.0 | Notifying role MASTER (preference 0) for term 2 to device:stratum-inventec-d10056-1
2021-12-17T15:15:22,334 | INFO | onos-flow-backup-0 | DeviceFlowTable | 192 - org.onosproject.onos-core-dist - 2.6.0 | Synchronizing device device:stratum-inventec-d10056-1 flows for term 2
2021-12-17T15:15:22,334 | INFO | onos-device-manager-background | StreamClientImpl | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.6.0 | Setting mastership on device:stratum-inventec-d10056-1... master=true, newElectionId=40, masterElectionId=null, sessionOpen=true
2021-12-17T15:15:22,406 | INFO | onos-pipeconf-watchdog-0 | PipelineConfigClientImpl | 237 - org.onosproject.onos-protocols-p4runtime-ctl - 2.6.0 | Setting pipeline config for device:stratum-inventec-d10056-1 to org.onosproject.foo.simple_l3_arping.tofino.montara_sde_9_4_0...
2021-12-17T15:15:22,631 | WARN | grpc-default-executor-1 | P4RuntimeClientImpl | 230 - org.onosproject.onos-protocols-grpc-ctl - 2.6.0 | Error while performing SET-pipeline-config on device:stratum-inventec-d10056-1: INVALID_ARGUMENT: Unknown format for p4_device_config.·

Stratum log snippet

bf-sde> I20211217 23:15:22.359282 12997 p4_service.cc:778] Controller (connection_id: 1, election_id: 20, uri: ipv4:10.4.16.144:55446) is connected as MASTER for node (aka device) with ID 1.
I20211217 23:15:22.359341 12997 p4_service.cc:778] Controller (connection_id: 1, election_id: 40, uri: ipv4:10.4.16.144:55446) is connected as MASTER for node (aka device) with ID 1.
E20211217 23:15:22.631534 14390 bf_pipeline_utils.cc:42] StratumErrorSpace::ERR_INVALID_PARAM: Unknown format for p4_device_config.
E20211217 23:15:22.631582 14390 bfrt_node.cc:153] Return Error: ExtractBfPipelineConfig(config, &bf_config) failed with StratumErrorSpace::ERR_INVALID_PARAM: Unknown format for p4_device_config.
E20211217 23:15:22.631603 14390 bfrt_switch.cc:74] Return Error: DoVerifyForwardingPipelineConfig(node_id, config) failed with StratumErrorSpace::ERR_INVALID_PARAM: Unknown format for p4_device_config.·
E20211217 23:15:22.631625 14390 error_buffer.cc:30] (p4_service.cc:456): Failed to set forwarding pipeline config for node 1: Unknown format for p4_device_config.·
E20211217 23:15:25.246840 14390 bf_pipeline_utils.cc:42] StratumErrorSpace::ERR_INVALID_PARAM: Unknown format for p4_device_config.
20211217 23:15:43.234520 12997 p4_service.cc:805] Controller (connection_id: 1, election_id: 40, uri: ipv4:10.4.16.144:55446) which was MASTER for node (aka device) with ID 1 is disconnected. The node is now orphan :(

I am guessing this has something to do with the formatting of the P4Info file?

Wondering if anyone else has encountered this issue before. Any debugging tips/guidance would be really helpful.

Thanks,

Prashanth

saki nanaki

unread,

May 16, 2022, 6:57:14 AM5/16/22

to ONOS Developers, Prashanth Iyengar

Hi, I face the same problem, I guess it's the The stratum of a higher version is incompatible with the netcfg.json file of an earlier version. Have you solve this problem? Thank you.

Davide Scano

unread,

May 16, 2022, 9:57:16 AM5/16/22

to ONOS Developers, saki nanaki, Prashanth Iyengar

Hi,

as you can see https://github.com/stratum/stratum/releases/tag/2021-12-13 stratum 2021 is compatible with SDE 9.5.0.

More in details, I think that the the p4 program should be compiled with the same SDE supported by stratum. Furthermore the nrtcofig.json file that you pushing use the "montara_sde_9_4_0" that is not supported by release 2021.

My tips are based on https://github.com/stratum/fabric-tna, this application works with the last version of stratum and both the p4 program and stratum have the same SDE version.

Best

Davide

Tamihiro Lee

unread,

May 17, 2022, 8:55:58 PM5/17/22

to ONOS Developers, 0ice...@gmail.com, saki nanaki, Prashanth Iyengar

Hi saki,

I have been through the same issue as yours.
Previously my labbing setup had been bf-sde-9.3.1, stratum-bf-21.06, and onos-2.5.1.
Then I needed to upgrade SDE to 9.7.0. In doing so I had to choose a compatible version of stratum and onos as Davide suggested.
My current choice is stratum-bfrt-22.03 and onos-2.7.0.
With stratum-bfrt you can no longer use the older tofino.bin and context.json, instead you need to generate a new format of device config binary this way.

https://github.com/stratum/stratum/blob/main/stratum/hal/bin/barefoot/README.pipeline.md#stratum-bfpipelineconfig-format-and-the-bfpipelinebuilder

Copy this device_config.pb.bin along with p4info.txt over to your onos app source directory.
And you need to edit PipeconfLoader.java to include them in the package.
I'd had no idea how I'd do it until fabric-tna was opensourced (thank goodness!) some time ago.

Below is the changes I've made, FYI.

@@ -37,9 +37,8 @@ import java.net.URL;
import java.util.List;
import java.util.stream.Collectors;

-import static org.onosproject.net.pi.model.PiPipeconf.ExtensionType.TOFINO_BIN;
-import static org.onosproject.net.pi.model.PiPipeconf.ExtensionType.TOFINO_CONTEXT_JSON;
import static org.onosproject.net.pi.model.PiPipeconf.ExtensionType.P4_INFO_TEXT;
+import static org.onosproject.net.pi.model.PiPipeconf.ExtensionType.RAW_DEVICE_CONFIG;

/**
@@ -51,8 +50,7 @@ public final class PipeconfLoader {
private final Logger log = LoggerFactory.getLogger(getClass());

private static final String P4INFO_PATH = "/p4info.txt";
- private static final String TOFINO_BIN_PATH = "/tofino.bin";
- private static final String TOFINO_CONTEXT_JSON_PATH = "/context.json";
+ private static final String TOFINO_DEVICE_CONFIG_PATH = "/device_config.pb.bin";

@Reference(cardinality = ReferenceCardinality.MANDATORY)
private PiPipeconfService pipeconfService;
@@ -84,8 +82,7 @@ public final class PipeconfLoader {
private PiPipeconf buildPipeconf() throws P4InfoParserException {

final URL p4InfoUrl = PipeconfLoader.class.getResource(P4INFO_PATH);
- final URL tofinoBinUrl = PipeconfLoader.class.getResource(TOFINO_BIN_PATH);
- final URL tofinoContextJsonUrl = PipeconfLoader.class.getResource(TOFINO_CONTEXT_JSON_PATH);
+ final URL tofinoDeviceConfigUrl = PipeconfLoader.class.getResource(TOFINO_DEVICE_CONFIG_PATH);
final PiPipelineModel pipelineModel = P4InfoParser.parse(p4InfoUrl);

return DefaultPiPipeconf.builder()
@@ -93,8 +90,7 @@ public final class PipeconfLoader {
.withPipelineModel(pipelineModel)
.addBehaviour(PiPipelineInterpreter.class, InterpreterImpl.class)
.addExtension(P4_INFO_TEXT, p4InfoUrl)
- .addExtension(TOFINO_BIN, tofinoBinUrl)
- .addExtension(TOFINO_CONTEXT_JSON, tofinoContextJsonUrl)
+ .addExtension(RAW_DEVICE_CONFIG, tofinoDeviceConfigUrl)
.build();
}

2022年5月16日月曜日 22:57:16 UTC+9 0ice...@gmail.com:

Tamihiro Lee

unread,

May 18, 2022, 9:33:27 AM5/18/22

to ONOS Developers, Tamihiro Lee, 0ice...@gmail.com, saki nanaki, Prashanth Iyengar

Wrt netcfg.json, I can't tell how yours is incompatible with newer versions of onos/stratum/bf-sde because mine hasn't changed except for the grpc port number and I have not faced any problems throughout version upgrades.

--- a/netcfg.json
+++ b/netcfg.json
@@ -2,7 +2,7 @@
"devices": {
"device:sw1": {
"basic": {
- "managementAddress": "grpc://198.18.1.1:28000?device_id=1",
+ "managementAddress": "grpc://198.18.1.1:9559?device_id=1",
"driver": "stratum-tofino",
"pipeconf": "tamil.bf-dnat-x2"
}