Rx Stream stats appearing as 0 for icmp stream though icmp response recieved by trex ns

[chris...@gmail.com]

Trex version : v3.00

Stateless

sudo ./t-rex-64 -i --software --cfg trex_cfg.yaml
Killing Scapy server... Scapy server is killed
Starting Scapy server..... Scapy server is started
Starting Cmds server.... Cmds server is started
The ports are bound/configured.
Starting  TRex v3.00 please wait  ...
 set driver name net_af_packet
 driver capability  : SLRO
 set dpdk queues mode to ONE_QUE
 Number of ports found: 2 (dummy among them: 1)
zmq publisher at: tcp://*:4500
 wait 1 sec .
port : 0
------------
link         :  link : Link Up - speed 10000 Mbps - full-duplex
promiscuous  : 0
 number of ports         : 2
 max cores for 2 ports   : 1
 tx queues per port      : 3
 -------------------------------
RX core uses TX queue number 65535 on all ports
 core, c-port, c-queue, s-port, s-queue, lat-queue
 ------------------------------------------
 1        0      0       1       0      0
 -------------------------------
Cleanup of old namespaces related to Linux-based stack
Cleanup Done

trex_cfg.yaml

### Config file generated by dpdk_setup_ports.py ###

- port_limit: 2
  version   : 2
  #c         : 2
  interfaces: ['test1', 'dummy']
  stack     : linux_based
  low_end   : true
  low_end_core: 3

ICMP Stream:

        packet = (Ether(src='00:00:00:01:02:03',dst='"dstmac") /
                  Dot1Q(vlan=1552) /
                  IP(src='25.20.20.100',proto=1,dst='25.20.20.10') /
                  ICMP() /
                  Raw(load='a' * 954))

        vm = STLVM()
        stream = STLStream(packet = STLPktBuilder(pkt = packet, vm = vm),stream_id = 1,name = 'TC16_UC2_Ping',mode =STLTXSingleBurst(total_pkts = 2,pps = 50),flow_stats = STLFlowStats(pg_id = 1))
        streams.append(stream)

Trex ns :

trex(service)>ns show-node -p 0 --mac 00:00:00:01:02:03

Setting port 0 in with namespace configuration               [SUCCESS]


wait_for_async_results                                       [SUCCESS]

{'nodes': [{'bpf': '',
            'ether': {'src': '00:00:00:01:02:03'},
            'ipv4': {'dst': '25.20.20.10', 'src': '25.20.20.100'},
            'ipv6': {'enabled': False},
            'is-shared-ns': False,
            'is_bird': False,
            'linux-ns': 'trex-a-0-1',
            'linux-veth-external': 'trex-a-0-1-T',
            'linux-veth-internal': 'trex-a-0-1-L',
            'vlan': {'tags': [1552], 'tpids': []},
            'vlan_filter': 'no_tcp_udp'}]}
trex(service)>

Capture in linux namespace:

ip netns exec trex-a-0-1 tcpdump -i any

13:35:52.703904 IP _gateway > trex-trex-0: ICMP echo reply, id 0, seq 0, length 962

Stats:

trex(service)>stats -s
Streams Statistics

  PG ID    |         1
-----------+------------------
Tx pps     |             0 pps
Tx bps L2  |             0 bps
Tx bps L1  |             0 bps
---        |
Rx pps     |             0 pps
Rx bps     |             0 bps
----       |
opackets   |                 2
ipackets   |                 0
obytes     |              2008
ibytes     |                 0
-----      |
opackets   |            2 pkts
ipackets   |            0 pkts
obytes     |           2.01 KB
ibytes     |               0 B

Can you please help me understand on what is going wrong here ? 

[chris...@gmail.com]

Can someone please help here? Am I missing something?

[Dominik Tran]

Did you check if packets actually come back? According to your configuration file, you have only 1 interface "test1". If you send packets from this interface, those packets should arrive back on the same interface. You have shown stream stats, but not port stats.

Dne pondělí 8. července 2024 v 9:32:32 UTC+2 uživatel chris...@gmail.com napsal:

[chris...@gmail.com]

Thanks much Dominik for your response . 

port stats i am unable to trace as there are some 1000 GTPU packets which are coming into my trex interface from some other setup which i am unable to stop . 

But i did a capture rx on port 0 on trex and couldn't see the icmp response. I could see the icmp response from SUT till trex's namespace.

[root@trex-trex-0 stl]# ip netns exec trex-a-0-1 tcpdump -i any
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes

05:29:53.443756 IP _gateway > trex-trex-0: ICMP echo reply, id 0, seq 0, length 962
05:29:53.443790 IP _gateway > trex-trex-0: ICMP echo reply, id 0, seq 0, length 962

Any idea why these packets are not visible within trex's capture? Could that be the reason that packets are not seen on stream stats as well ?

What more debugging can be done from my side?

[Dominik Tran]

If you don't see packets on RX capture I would expect that you would also not see them in stream stats. So I would try to troubleshoot that first.

If you are receiving other unrelated packets, you could increase number of generated packets (maybe 10kpps) just to check if you see increased RX traffic on port stats. But since you're seeing packets in netspace, I would expect that TRex processes them.

You are using 'linux_based' stack. Documentation mentions that 'ARPs/pings are redirected to Linux interfaces, that deal with them. The Linux interfaces reside in separate network namespace'. Maybe TRex redirects your ICMP packets to kernel without any other processing? I am not familiar with linux_based stack in TRex, so I can't help. You could also try generating some IP/UDP traffic to see if TRex handles it differently than ARP/ICMP traffic.

Dne úterý 9. července 2024 v 7:41:26 UTC+2 uživatel chris...@gmail.com napsal:

[chris...@gmail.com]

Hi Dominik , 

Yes, i can see the rate increment in port stats . 

I am sending at the rate of 10Mbps and i saw the total rx rate increasing by 10 Mbps

Before sending icmp:

Global Statistics

connection   : localhost, Port 4501                       total_tx_L2  : 0 bps
version      : STL @ v2.90                                total_tx_L1  : 0 bps
cpu_util.    : 0.0% @ 1 cores (1 per dual port)           total_rx     : 65.14 Mbps
rx_cpu_util. : 0.0% / 8.26 Kpps                           total_pps    : 0 pps
async_util.  : 0% / 8.53 bps                              drop_rate    : 0 bps
total_cps.   : 0 cps                                      queue_full   : 0 pkts

After sending icmp :

trex(service)>stats
Global Statistics

connection   : localhost, Port 4501                       total_tx_L2  : 9.77 Mbps
version      : STL @ v2.90                                total_tx_L1  : 9.96 Mbps
cpu_util.    : 0.0% @ 1 cores (1 per dual port)           total_rx     : 74.73 Mbps
rx_cpu_util. : 0.0% / 9.44 Kpps                           total_pps    : 1.22 Kpps
async_util.  : 0% / 7.19 bps                              drop_rate    : 0 bps
total_cps.   : 0 cps                                      queue_full   : 0 pkts

So , it is seen in port stats and not in stream stats .Also , trex's console capture is not showing it .

Can someone please help ?

Thanks,

Chris.

[chris...@gmail.com]

Other streams like GTPU stats are working fine.Does it mean that icmp stats doesn't work ?