TRex ASTF & L7 transaction stats in traffic/protocol mix

[Petr Redkin]

Hello,

Does TRex ASTF supports displaying L7 transaction statistics (total / success / failure) in a traffic / protocol mix test (EMIX / sfr) or an architectural approach with using a polling model prevented to implement this important counter?

I know that ASTF statistics exist, but these are low level tcp / udp stats that does not indicate the ultimate success / failure of the transaction (for example, a client-server transaction can be successful despite multiple retransmissions).

Transaction statistics are collected by spirent / ixia and are required in "serious" performance testing methodologies (e.g. netsecopen, nss labs).

Best Regards, 

Petr Redkin

[hanoh haim]

Hi Petr,

I think this is a good suggestion. We have a TCP/UDP (transport) counters and any error in the application layer will affect those counters.   

For example, if a DNS/UDP response has a timeout the flow will be closed using keepalive and there would be an error indication.

In our main use-case we are less interested to test the application layer errors, we are more interested to test a Network Gear that save context of flow/client/server and manipulate those information for example DPI/NAT/SD-WAN and in those feature case we are not expecting that the network gear will introduce an only an application error and not in the transport layer so we focus on transport counters. 

If you can give a specific application use-case and counter that your DUT is manipulated it will help. 

Thanks

Hanoh

[Petr Redkin]

Thank you for the answer, Hanoh.

Estimation approach based on L7 transactions is a good way for a reason - we measure what definitely affects the end user.

Decent TCP stack successfully mitigate a lot of detected problems and end user actually will not be affected (or will be affected in way that we can measure on L7) when a transport anomaly will be detected.

For example in a real test we encountered TCP duplicated packets when testing a DUT (even in a low rate):

1) If we use approach with reacting to every transport counter - that is a serious issue and performance results will be zero.

2) If we use approach with reacting only to L7 counters (what is really matters) - that is not such a serious issue if transactions is good (successful, not delayed) and duplicated packets will really affects L7 counters because we will see a penalty in L7 throughput.

Best Regards, 

Petr Redkin

вторник, 13 апреля 2021 г. в 08:49:21 UTC+3, Hanoch Haim:

[hanoh haim]

Hi Petr, 

There are two levels of issues in TCP counters, one level is solvable, like out of order or a small number of drops the application layer won't be affected. There are more sever issues that create drop of service like closing a flow in keepalive or termination due to resource issues. 

The application level counters would be good for DUT that has a deep knowledge of the application and change the L7 data in a way that keep the application works (like TLS proxy with advance change of http2/fields) . Network gear does not do that so there is no need to get into the application layer. 

You can always look into the transport counters and decided what is your criteria  (assuming your customer is aligned with this criteria)

Thanks

Hanoh

[Petr Redkin]

Haim, thank you for explanation and TRex itself.

I understood your approach, but it is not standard for security device testing (NGFW, NIPS/NIDS, BDS, etc). As I mention early netsecopen/NSS labs used transaction stats as a indicator of success/failure.

среда, 14 апреля 2021 г. в 11:12:40 UTC+3, Hanoch Haim: