Packets are all dropped when the P4 program length is long

[Jiarong Xing]

Hi there, 

I am using P4-16 to program a Netronome Agilio CX NIC. I am testing the throughput using a straightline program. I found that when the number of tables is longer than a number, all packets are suddenly dropped. For example, the first x tables in my program are all the same (match on 4-tuple and decrement ttl, with different rules), and the last table forwards packets to VFs.  If x=19, I can packets are received on the VF, but if x=20, all packets are dropped. As the following counters show:

sudo /opt/netronome/bin/nfp-rtsym _pif_counter_RX
0x0000000000:  0x00752fed 0x00000000

sudo /opt/netronome/bin/nfp-rtsym _pif_counter_DROP*
_pif_counter_DROP_MCAST_PORT_OUT_BOUNDS: 0x00000000 0x00000000
_pif_counter_DROP_META: 0x00000000 0x00000000
_pif_counter_DROP_NFD_NO_CREDITS: 0x00000000 0x00000000
_pif_counter_DROP_INVALID_PKT_MOD_SCRIPT: 0x00000000 0x00000000
_pif_counter_DROP_PORT_PAUSED: 0x00000000 0x00000000
_pif_counter_DROP: 0x007531c0 0x00000000

I also checked the ERROR info.

_pif_counter_ERROR_NFDMETA: 0x00000000 0x00000000
_pif_counter_ERROR_FLOWCACHE: 0x00000000 0x00000000
_pif_counter_ERROR_PARREP_CUSTOM: 0x00000000 0x00000000

_pif_counter_ERROR_PARREP i24.emem addr=0x00008084d0 size=0x0200
*

_pif_counter_ERROR_DEPARSE i24.emem addr=0x0000808710 size=0x0028
0x0000000020:  0x00000000 0x00000000

_pif_counter_ERROR_EGRESS i24.emem addr=0x0000808790 size=0x0010
*

_pif_counter_ERROR_INGRESS i24.emem addr=0x00008087a0 size=0x0010
0x0000000000:  0x00000000 0x00000000 0x0075320b 0x00000000

_pif_counter_ERROR_WAIT_BLM_BUF_ALLOC: 0x00000000 0x00000000
_pif_counter_ERROR_WAIT_CTM_PKT_ALLOC: 0x00000000 0x00000000
_pif_counter_ERROR_PKTMETA: 0x00000000 0x00000000

I didn't understand how could this happen. Hope I can find some help here. Thanks!

[Maksymilian Czudziak]

Hi,
Maybe you should check how rules are inserted, I observed that when there are no rules for the table, the table drops packets.
How do you insert the .nffw file to the NIC and pif_config.json? I found out that when rules are inserted using rtecli tool after flashing the card, they sometimes have to be inserted twice to be effective.
I'd like to propose that you also check using the registers which tables are matched and which are not.
Regards, 
Maksymilian

[Jiarong Xing]

Hi, thanks for your reply! I did more debugging on this. I found that all table entries are installed. This time I switched to using exact tables each only has one default rule. But still, the 19th can be hit, the 20th sees no packets. Is there hard constraint on this?

[Maksymilian Czudziak]

There is no constraint that I am aware of, but many P4_16 pieces of code work on Agilio unstably that I'm not surprised. 
Maybe you should try switching to P4_14?