Export of host TCP performance metrics

[Peter Phaal]

The Linux TCP stack maintains detailed network performance metrics for each open connection that can be retrieved using the tcp_diag kernel module. 

The following extended flow structure allows this information to be attached to packet samples:

/* TCP connection state */
/* Based on Linux struct tcp_info */
/* opaque = flow_data; enterprise=0; format=2209 */
struct extended_tcp_info {
  packet_direction dir;     /* Sampled packet direction */
  unsigned int snd_mss;     /* Cached effective mss, not including SACKS */
  unsigned int rcv_mss;     /* Max. recv. segment size */
  unsigned int unacked;     /* Packets which are "in flight" */
  unsigned int lost;        /* Lost packets */
  unsigned int retrans;     /* Retransmitted packets */
  unsigned int pmtu;        /* Last pmtu seen by socket */
  unsigned int rtt;         /* smoothed RTT (microseconds) */
  unsigned int rttvar;      /* RTT variance (microseconds) */
  unsigned int snd_cwnd;    /* Sending congestion window */
  unsigned int reordering;  /* Reordering */
  unsigned int min_rtt;     /* Minimum RTT (microseconds) */
}

The Host sFlow agent (http://www.sflow.net) has been extended to export this data and sflowtool (https://github.com/sflow/sflowtool) has been extended to decode and display the metrics.

The following article provides additional detail: http://blog.sflow.com/2016/10/network-performance-monitoring.html

[rick jones]

Peter (et al) -

What is the minimum Linux kernel version with support for those additional metrics?

happy benchmarking,

rick jones

[Peter Phaal]

The inet_diag module looks like it first arrived in the 3.11 kernel.
The random sampling function in BPF looks like it was first
implemented in 3.18.

The tcp_info lookup is very efficient, you don't need to walk the
table to find a record, you can ask for a specific record by filling
in a tcp socket structure. The randomly sampled packet headers provide
the information needed to make the request.

You might also be interested to know that your recommendations for
scheduling counter export have been implemented by the Host sFlow and
Open vSwitch, e.g.
http://openvswitch.org/pipermail/dev/2016-August/078743.html

Counter export from fast changing Docker container pools also posed an
interesting challenge:
http://blog.sflow.com/2016/09/asynchronous-docker-metrics.html

[rick jones]

> You might also be interested to know that your recommendations for
> scheduling counter export have been implemented by the Host sFlow and
> Open vSwitch, e.g.
> http://openvswitch.org/pipermail/dev/2016-August/078743.html

Excellent!

happy benchmarking,

rick

>
> Counter export from fast changing Docker container pools also posed an
> interesting challenge:
> http://blog.sflow.com/2016/09/asynchronous-docker-metrics.html
>

> --
> You received this message because you are subscribed to the Google Groups "sFlow" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sflow+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Vrushali Bhutada]

After installing hsflow on mininet, i am trying to fetch extended_tcp_info for traffic through ovs-switch. Though i dont see these stats coming in to sflow-rt server. I am seeing other host sflow stats like CPU disk but but extended_tcp_info (tcp_rtt) flow keys being pushed to sflow-rt server for tcp traffic. How to get these stats working in sflow-rt?

[Peter Phaal]

Mininet creates a separate network name space for each emulated host. The TCP state information is only accessible within each name space and not within the root namespace where the Host sFlow agent and Open vSwitch agents are running.

You could try running an instance of Host sFlow within each emulated host's name space. You would need to enable the Host sFlow PCAP module (https://sflow.net/host-sflow-linux-config.php) to sample packets on the veth devices within emulated host name space.

If you need additional information, then please direct questions on the Mininet (https://mailman.stanford.edu/mailman/listinfo/mininet-discuss) or Host sFlow (https://groups.google.com/forum/#!forum/host-sflow) mailing lists where they are more likely to find an audience with relevant experience.