Issues with Initializing an Agilio CX 2x10GbE via sdk_install.sh

[Sean Choi]

Hello,

I recently got a new server and a set of Agilio CX cards.

Using the upstream drivers and firmware, as well as the netronome repoy drivers and firmware (agilio-nic-firmware, agilio-nfp-driver-dkms), I was able to see the interfaces route basic traffic.

Now, I was hoping to install nfp-sdk-p4-rte-6.1.0.1-preview-3214.ubuntu.x86_64.tgz and nfp-sdk_6.1.0.1-preview-3243-2_amd64.deb into the system.

I tried various methods such as

1. dpkg -i nfp-sdk_6.1.0.1-preview-3243-2_amd64.deb followed by sdk6_rte_install.sh install

2. Clean up all agilio-* and nfp-* packages from dpkg --list and reinstall sdk6_rte_install.sh install_force_bsp

3. Uninstalled via sdk6_rte_install.sh uninstall_with_bsp

But no interfaces can be found in both ip link and ifconfig

After reboot, I get the following result from dmesg | grep nfp. Any rmmod nfp;  modprobe nfp nfp_dev_cpp=1 nfp_pf_netdev=0  combination results in the same output.

[    7.826495] nfp: loading out-of-tree module taints kernel.

[    7.826691] nfp: module verification failed: signature and/or required key missing - tainting kernel

[    7.831737] nfp: NFP PCIe Driver, Copyright (C) 2014-2017 Netronome Systems

[    7.831739] nfp src version: no-src-ver (o-o-t)

               nfp src path: /var/lib/dkms/nfp-bsp/2018.08.17.1104/build/

               nfp build user id: root

               nfp build user: root

               nfp build host: mendel1

               nfp build path: /var/lib/dkms/nfp-bsp/2018.08.17.1104/build

[    7.832130] nfp-net-vnic: NFP vNIC driver, Copyright (C) 2010-2015 Netronome Systems

[    7.833118] nfp 0000:5e:00.0: Netronome Flow Processor NFP4000/NFP6000 PCIe Card Probe

[    7.833251] nfp 0000:5e:00.0: RESERVED BARs: 0.0: General/MSI-X SRAM, 0.1: PCIe XPB/MSI-X PBA, 0.4: Explicit0, 0.5: Explicit1, free: 20/24

[    7.834049] nfp 0000:5e:00.0: Model: 0x40010010, SN: 00:15:4d:13:81:bd, Ifc: 0x10ff

[    7.848242] nfp 0000:5e:00.0: Assembly: SMCAMDA0096-000117290645-11 CPLD: 0x1030000

[    7.849785] nfp 0000:5e:00.0: nfp: not registering HWMON (no NSP info)

Then, I ran rmmod nfp; insmod /var/lib/dkms/nfp-bsp/2018.08.17.1104/build/nfp.ko nfp_dev_cpp=1 and got the following output.

Which seems to suggest no firmware file is found? Doesn't sdk6_rte_install.sh install the firmware by itself?

[  201.259059] nfp: NFP PCIe Driver, Copyright (C) 2014-2017 Netronome Systems

[  201.259062] nfp src version: no-src-ver (o-o-t)

               nfp src path: /var/lib/dkms/nfp-bsp/2018.08.17.1104/build/

               nfp build user id: root

               nfp build user: root

               nfp build host: mendel1

               nfp build path: /var/lib/dkms/nfp-bsp/2018.08.17.1104/build

[  201.259101] nfp-net-vnic: NFP vNIC driver, Copyright (C) 2010-2015 Netronome Systems

[  201.259608] nfp 0000:5e:00.0: Netronome Flow Processor NFP4000/NFP6000 PCIe Card Probe

[  201.259657] nfp 0000:5e:00.0: RESERVED BARs: 0.0: General/MSI-X SRAM, 0.1: PCIe XPB/MSI-X PBA, 0.4: Explicit0, 0.5: Explicit1, free: 20/24

[  201.259714] nfp 0000:5e:00.0: Model: 0x40010010, SN: 00:15:4d:13:81:bd, Ifc: 0x10ff

[  201.263805] nfp 0000:5e:00.0: Assembly: SMCAMDA0096-000117290645-11 CPLD: 0x1030000

[  201.487296] nfp 0000:5e:00.0: BSP: 01011b.01011b.0100ff

[  201.487299] nfp 0000:5e:00.0: nfp: Looking for firmware file in order of priority:

[  201.488510] nfp 0000:5e:00.0: nfp:   netronome/serial-00-15-4d-13-81-bd-10-ff.nffw: not found

[  201.488524] nfp 0000:5e:00.0: nfp:   netronome/pci-0000:5e:00.0.nffw: not found

[  201.488667] nfp 0000:5e:00.0: nfp:   netronome/nic_AMDA0096-0001_2x10.nffw: not found

[  201.489074] nfp 0000:5e:00.0: nfp: Failed to find data vNIC memory symbol

[  201.489265] nfp 0000:5e:00.0: Netronome NFP Fallback driver

Here is an output of ls -ogR --time-style="+" /lib/firmware/netronome/ and can see nic_AMDA0096-0001_2x10.nffw

lrwxrwxrwx 1 64  nic_AMDA0058-0011_2x40.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0058-0011_2x40.nffw

lrwxrwxrwx 1 64  nic_AMDA0058-0012_2x40.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0058-0012_2x40.nffw

lrwxrwxrwx 1 65  nic_AMDA0078-0011_1x100.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0078-0011_1x100.nffw

lrwxrwxrwx 1 64  nic_AMDA0081-0001_1x40.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0081-0001_1x40.nffw

lrwxrwxrwx 1 64  nic_AMDA0081-0001_4x10.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0081-0001_4x10.nffw

lrwxrwxrwx 1 64  nic_AMDA0096-0001_2x10.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0096-0001_2x10.nffw

lrwxrwxrwx 1 64  nic_AMDA0097-0001_2x40.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0097-0001_2x40.nffw

lrwxrwxrwx 1 69  nic_AMDA0097-0001_4x10_1x40.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0097-0001_4x10_1x40.nffw

lrwxrwxrwx 1 64  nic_AMDA0097-0001_8x10.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0097-0001_8x10.nffw

lrwxrwxrwx 1 69  nic_AMDA0099-0001_1x10_1x25.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0099-0001_1x10_1x25.nffw

lrwxrwxrwx 1 64  nic_AMDA0099-0001_2x10.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0099-0001_2x10.nffw

lrwxrwxrwx 1 64  nic_AMDA0099-0001_2x25.nffw -> /opt/netronome/agilio-sriov-firmware/nic_AMDA0099-0001_2x25.nffw

 

I also checked to see if BSP is installed via ./sdk6_rte_install.sh bsp_version_info resulting in.

 NFP-BSP release packaged with this installation: nfp-bsp_2018.08.17.1104-1

 Currently installed NFP-BSP packages:

 No NFP-BSP currently installed

nfp-hwinfo shows the following result

nfp.interface=pci.0.0

nfp.model=0x40010010

nfp.serial=00:15:4d:13:81:33

assembly.revision=11

assembly.model=lithium

assembly.partno=AMDA0096-0001

assembly.serial=17291079

assembly.vendor=SMC

ddr0.spd=spi:1:0:0x3F0F00

ddr1.spd=spi:1:0:0x3F0F00

ddr2.spd=none

ddr3.spd=none

ddr4.spd=none

ddr5.spd=none

emu1.type=cache

emu2.type=cache

ethm.mac=00:15:4d:13:81:33

eth.mac=00:15:4d:13:81:34

eth.macs=2

pcie0.type=ep

chip.model=NFP4001

chip.revision=B0

chip.model.device=0x62004520

chip.identifier=0xd78fca145

chip.model.hard=0x5

chip.model.soft=0x40010096

chip.route=0xc96f1e8e

chip.island=0x1001f13000112

core.speed=633

me.speed=633

arm.speed=475

nfp-boot.version= ()

bsp.version.primary=01011b

bsp.version.secondary=01011b

flash.data.bus=1

ddr0.mem.size=1024

ddr1.mem.size=1024

ddr0.mem.speed=1600

ddr1.mem.speed=1600

emu0.mem.size=2048

emu0.mem.base=0x2000000000

emu1.mem.size=3

emu1.mem.base=0x9900000000

emu2.mem.size=0

emu2.mem.base=0x0

arm.mem.size=96

arm.mem.base=0x207a000000

cpld.location=spi:2:2:4

pmon.limit=25.0

pmon.12v=cpld:7:I:32_0:0.00249:0

pmon.3v3=static:0.54

phy0.label=0

phy0.nbi=0

phy0.port=0

phy0.lanes=1

phy0.sff=8431

phy0.pin.link=-cpld:2:2:0xd.0

phy0.pin.activity=-cpld:2:2:0xd.3

phy0.SFF-8431=ee1:0:0x50:0x0

phy0.SFF-8472=ee1:0:0x51:0x0

phy0.pin.present=-cpld:2:2:0x9.5

phy0.pin.rate_select_0=cpld:2:2:0x9.2

phy0.pin.rate_select_1=cpld:2:2:0x9.1

phy0.pin.tx_disable=cpld:2:2:0x9.4

phy0.pin.tx_fault=cpld:2:2:0x9.12

phy0.pin.rx_los=cpld:2:2:0x9.0

phy0.type=SFP+

phy0.media=X

eth0.media=X

eth0.label=0.0

eth0.phy=0

eth0.lane=0

eth0.lanes=1

eth0.boot=1

phy1.label=1

phy1.nbi=0

phy1.port=4

phy1.lanes=1

phy1.sff=8431

phy1.pin.link=-cpld:2:2:0xd.4

phy1.pin.activity=-cpld:2:2:0xd.7

phy1.SFF-8431=ee1:1:0x50:0x0

phy1.SFF-8472=ee1:1:0x51:0x0

phy1.pin.present=-cpld:2:2:0x9.11

phy1.pin.rate_select_0=cpld:2:2:0x9.8

phy1.pin.rate_select_1=cpld:2:2:0x9.7

phy1.pin.tx_disable=cpld:2:2:0x9.10

phy1.pin.tx_fault=cpld:2:2:0x9.13

phy1.pin.rx_los=cpld:2:2:0x9.6

phy1.type=SFP+

phy1.media=X

eth4.media=X

eth4.label=1.0

eth4.phy=1

eth4.lane=0

eth4.lanes=1

eth4.boot=1

phy0.ledblink=cpld:2:2:0xd.10

phy1.ledblink=cpld:2:2:0xd.12

cpld.version=0x1030000

eth0.mac=00:15:4d:13:81:34

eth4.mac=00:15:4d:13:81:35

board.state=15

bootloader.version=default (e3136d5f74c39ed9b039cfccfb53a30b86d60cbb)

bsp.version=01011b.01011b.0100ff

I am completely unsure what to do next. I have attached a result for the nfp-troubleshoot script as well.

Please advise.

Thank you so much!

[Sean Choi]

One Update:

I upgraded the kernel to 4.15, and reinstalled everything.

Running insmod /var/lib/dkms/nfp-bsp/2018.08.17.1104/4.15.0-39-generic/x86_64/module/nfp.ko nfp_dev_cpp=1 brings the interfaces back.

However, ./sdk6_rte_install.sh bsp_version_info still gives no BSP installed

Thank you

[Sean Choi]

Any attempt to run systemctl start nfp-sdk6-rte results in the following.

RTE: 2018-11-18 22:38:36:183 - pif_log.c:162 log level set to INFO (4)

RTE: 2018-11-18 22:38:36:183 - nfp_pif.c:45 Attaching new client to device 0

RTE: 2018-11-18 22:38:36:188 - nfp_pif.c:85 Incompatible firmware detected

RTE: 2018-11-18 22:38:36:188 - pif_rte.cpp:425 Unable to attach to device 0: No such file or directory

RTE: 2018-11-18 22:38:36:188 - pif_rpc.cpp:1688 Opening RPC server. IP: 0.0.0.0 Port: 20206 User zlib: 1

RTE: 2018-11-18 22:38:36:188 - pif_rte.cpp:471 NFP P4 run time interface server v6.1.0.1-preview running

RTE: 2018-11-18 22:38:36:188 - pif_rte.cpp:473 Copyright (C) 2015-2018 Netronome Systems, Inc.  All rights reserved.

RTE: 2018-11-18 22:38:36:194 - nfp_pif.c:150 Invalid handle

On Monday, November 19, 2018 at 11:12:57 AM UTC+9, Sean Choi wrote:

[Helgard van Rensburg]

Hi Sean

Sorry this is a bug in the install script that has now been fixed for next release.

I know Gerhard has been in communication with you in regards to the installation of SDK RTE and setup if the Agilio SmartNICs for SDK work.

Are all your issues solved now or are there any installation/RTE related issues I can assist with or clarify?

Hegard van Rensburg

[Sean Choi]

All issues are resolved now.

[Marinos Dimolianis]

How did you manage to resolve it finally?

[Gerhard de Klerk]

Hi Marinos,

Please provide me with some more info regarding your setup, what errors you are encountering and what you are trying to achieve.

My first recommendation is to ensure that you are using the latest SDK and RTE as available on the support site.

Best regards,
Gerhard

--
You received this message because you are subscribed to the Google Groups "open-nfp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-nfp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-nfp/0b250bf7-fc53-4de7-9f02-cd433b0eab47%40googlegroups.com.

[Zhen Ni]

I'm currently having exactly the same issue when setting up SDK on ubuntu 16. Can you please share the solution to fix this problem?

[belma.tur...@gmail.com]

I think I am currently having the same issue (i.e., same command outputs as described). Could you please share how to fix this problem?

Best, 

Belma

[belma.tur...@gmail.com]

I got mine in the same way as before, by contacting their support. But it was months ago  (27/07/2020) 

Best,

Belma

On Friday, January 22, 2021 at 6:53:11 PM UTC+1 dipjyot...@gmail.com wrote:

Hello Folks,

How are you guys getting p4 sdk license. As far as I know, Netronome stopped releasing or supporting p4.

Regards,

Dip

--

You received this message because you are subscribed to the Google Groups "open-nfp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-nfp+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-nfp/b9d3f379-87e0-4cca-bf4d-21acd6408834n%40googlegroups.com.

[Octavio Herrera-Ruiz]

Addressing the latest posts (i.e., Jan 2021):    For Licenses, you do need to reach out to Netronome Support (he...@netronome.com or https://help.netronome.com). Please include your SmartNIC' S/N in your inquiry. Upon confirming your SmartNIC' S/N, you get access to the SDK, which includes a specific kernel module (nfp.ko) version. You are encouraged to remove any other versions present in your system and only use the one included with the SDK.

Octavio H.