Flow Monitor for TCP/UDP Tx/RxPackets and packet delivery ratio measurements

[Jan H]

Hi guys, 

maybe I have a total misunderstanding of how the flow monitor in NS-3 works. I try to measure the packet delivery ratio of TCP in a wireless single hop testbed. The channel is set up to lose approximately 50% of all packets. I use OLSR for routing. 

First of all I decided to test the channel with simple UDP transmissions, I used the UdpEchoServer/Client and their Tx/Rx traces to measure the sent and recieved packets. If I send 1000 packets, the server will recieve 494 packets, which is ok and the same number of packets as recognized by the pcap file. 

But the flow monitor tells me there are 1030 packets sent and 510 recieved?

If I try to deploy this on a TCP flow with 1000 bytes to send, flow monitor tells me there are 9 packets sent and 9 packets recieved. A look in the pcap files tells me there are 10 TCP packets sent on the client side and 5 packets recieved on the server side (I used the tcp bulk sender application). 

I used the flow monitor code found in https://groups.google.com/forum/#!topic/ns-3-users/e63hi63K8CU and some different examples. 

So my question is, do I missunderstand how flow monitor works? Or is there another mistake in my code? Why is the TCP PDR allways 100%? I need some code which measure all TCP packets leaving the interfaces on the client and are received on the server side (also retransmissions, ACKs, FINs, SYNs). 

Greetings

Jan

p.s. my code:

        std::map<FlowId, FlowMonitor::FlowStats> stats = flow_monitor->GetFlowStats ();

uint32_t txPacketsum = 0;

        uint32_t rxPacketsum = 0;

        uint32_t DropPacketsum = 0;

        uint32_t LostPacketsum = 0;

        double Delaysum = 0;

        for (std::map<FlowId, FlowMonitor::FlowStats>::const_iterator i = stats.begin (); i != stats.end (); ++i)

        {

                txPacketsum += i->second.txPackets;

                rxPacketsum += i->second.rxPackets;

                LostPacketsum += i->second.lostPackets;

                DropPacketsum += i->second.packetsDropped.size();

                Delaysum += i->second.delaySum.GetSeconds();

        }

        std::cout << "  All Tx Packets: " << txPacketsum << "\n";

        std::cout << "  All Rx Packets: " << rxPacketsum << "\n";

        std::cout << "  All Delay: " << Delaysum / txPacketsum <<"\n";

        std::cout << "  All Lost Packets: " << LostPacketsum << "\n";

        std::cout << "  All Drop Packets: " << DropPacketsum << "\n";

        std::cout << "  Packets Delivery Ratio: " << ((rxPacketsum *100) / txPacketsum) << "%" << "\n";

        std::cout << "  Packets Lost Ratio: " << ((LostPacketsum *100) / txPacketsum) << "%" << "\n";

[Konstantinos]

Hi Jan,

Please find my replies inline.

Regards,

K. 

On Friday, December 18, 2015 at 12:41:23 PM UTC, Jan H wrote:

Hi guys, 

maybe I have a total misunderstanding of how the flow monitor in NS-3 works. I try to measure the packet delivery ratio of TCP in a wireless single hop testbed. The channel is set up to lose approximately 50% of all packets. I use OLSR for routing. 

If you are working on single hop, there is no need for OLSR. That should not have an impact on the results though, just for reference.

 

First of all I decided to test the channel with simple UDP transmissions, I used the UdpEchoServer/Client and their Tx/Rx traces to measure the sent and recieved packets. If I send 1000 packets, the server will recieve 494 packets, which is ok and the same number of packets as recognized by the pcap file. 

 

But the flow monitor tells me there are 1030 packets sent and 510 recieved?

As you have already mentioned, UDP shows a relatively accurate result, ie half of the packets being dropped from the channel.

The difference from the packet number I would guess it is due to the OLSR messages and/or the return packets, i.e. echos. You said you are using UdpEchoServer client. 

The client sends 1000 and the server would reply that he received 1000 (or as many as he actually has received).

However in your FlowMonitor calculations you do not filter the results for those flows. You aggregate ALL flows into your statistics, either from client to server, or server to client or any other flows that might exist (OLSR traffic). 

In your  for loop you can have control on the packets to be counted, e.g.

for (std::map<FlowId, FlowMonitor::FlowStats>::const_iterator i = stats.begin (); i != stats.end (); ++i)
{

    Ipv4FlowClassifier::FiveTuple t = classifier->FindFlow (i->first);

    if (t.destinationAddress == Ipv4Address("10.10.10.10")){
    // only data with destination address 10.10.10.10 will be printed..

    }

}     

 

If I try to deploy this on a TCP flow with 1000 bytes to send, flow monitor tells me there are 9 packets sent and 9 packets recieved. A look in the pcap files tells me there are 10 TCP packets sent on the client side and 5 packets recieved on the server side (I used the tcp bulk sender application). 

Here you send 1000 bytes, not 1000 packets. These will be formatted into packets/segments with size depending on the TCP MTU. 

Also, note that TCP is a reliable protocol. So, it will try to retransmit a packet that was not successfully received. Hence the PCAP (NetDevice) may show losses, but at the Application layer you shouldn't see.

Similarly to the UDP calculations, you should filter the data for uplink/downlink (if needed), as you will count both ACK packets.

 

I used the flow monitor code found in https://groups.google.com/forum/#!topic/ns-3-users/e63hi63K8CU and some different examples. 

For the calculation of the metrics I would suggest to study the presentation of FlowMonitor here http://telecom.inesctec.pt/~gjc/flowmon-presentation.pdf

For example the Delay should be calculated based on the RxPackets, not on the Tx.

So my question is, do I missunderstand how flow monitor works? Or is there another mistake in my code?

Some minor misunderstandings are pointed out.  

 

Why is the TCP PDR allways 100%?

As said above, TCP is a reliable protocol and 100% reception is to be expected.

[Jan H]

Hi Konstantinos, 

thanks for your fast reply. 

First of all I use OLSR because I want to expand the scenario to an multihop network later. I mentioned that because this is the main point why I can not use IPv4 traces to measure the PDR of my transmission. It would also measure the OLSR packages, due to the fact that these are IP encapsulated. 

My goal is to measure all TCP packages of on flow. I thought the flow monitor is able to count this - maybe my implementation is wrong, I will check this. Thanks for your example!!! :)

To use the traces from application layer is no practical solution for me, because as you mentioned, the PDR will allways be 100% due to the reliable functionality from TCP. But one layer below this can not be correct, because TCP has to send much more packages as recieved due to the losses on the channel as can be seen in the PCAP files. I think the PDR should be below 50% and depending on how good the rate / congestion control of TCP/MAC will work.

So my question is, on which layer the flow monitor works? And why there are differences to the pcap files (in UDP and TCP)? By the way, of course I know that TCP will send a data stream and not packages. Therefore I have spoken of bytes to send. This results in 10 TCP packages (SYN, ACKs, FINs, retransmissions and normal ones) on the source side in my simulation (I counted in the pcap file). But the flow monitor tells me 9 packages? It's confusing, isn't it? 

An Additional question, why is there no tracing option in NS3 for TCP packages (Rx/Tx) like UDP / IP?

Thanks a lot!

Jan

[Jan H]

Hi again, 

I could improve my example and filter the flows. In fact, now UDP works well and all Rx/Tx packages counted correct. But the TCP packet count is still confusing me. I did the same implementation as in UDP. Here are my results from flow monitor xml-file and pcap file: 

TCP Flow number 1: here you can see there are 5 packages counted as tx and rx.

<?xml version="1.0" ?>

<FlowMonitor>

  <FlowStats>

    <Flow flowId="1" timeFirstTxPacket="+700000000000.0ns" timeFirstRxPacket="+700000042100.0ns" timeLastTxPacket="+700005343000.0ns" timeLastRxPacket="+700008183100.0ns" delaySum="+6302698.0ns" jitterSum="+7326400.0ns" lastDelay="+2840100.0ns" txBytes="1264" rxBytes="1264" txPackets="5" rxPackets="5" lostPackets="0" timesForwarded="0">

      <delayHistogram nBins="3" >

        <bin index="0" start="0" width="0.001" count="3" />

        <bin index="2" start="0.002" width="0.001" count="2" />

      </delayHistogram>

      <jitterHistogram nBins="3" >

        <bin index="0" start="0" width="0.001" count="1" />

        <bin index="1" start="0.001" width="0.001" count="1" />

        <bin index="2" start="0.002" width="0.001" count="2" />

      </jitterHistogram>

      <packetSizeHistogram nBins="30" >

        <bin index="2" start="40" width="20" count="3" />

        <bin index="25" start="500" width="20" count="1" />

        <bin index="29" start="580" width="20" count="1" />

      </packetSizeHistogram>

      <flowInterruptionsHistogram nBins="0" >

      </flowInterruptionsHistogram>

    </Flow>

For reasons of clarity I've uploaded the screenshots from wireshark for the sink and the source node. Here you can see, the source node sents 10 packages and the sink node just recieved 5 of them. It looks as if the Rx value is always correct. I have validated this with several measurements of different sizes. But the Tx value is not correct. Why it should be the same as Rx? (just for information, I have sorted the output by source IP)

Source Node:

Sink Node:

Any ideas why this happens? Unfortunately I have found nothing in the Internet or in the flow monitor documentation. =(

p.s. this should be the interesting part of my code-file:

....
}else{

BulkSendHelper source ("ns3::TcpSocketFactory",

InetSocketAddress (interfaces.GetAddress (sinkNode), 80));

// Set the amount of data to send in bytes.  Zero is unlimited.

source.SetAttribute ("MaxBytes", UintegerValue (maxBytes));

ApplicationContainer sourceApps = source.Install (nodes.Get (sourceNode));

sourceApps.Start (tcp_send_start);

sourceApps.Stop (Seconds (2000.0));

PacketSinkHelper sink ("ns3::TcpSocketFactory",

  InetSocketAddress (Ipv4Address::GetAny (), 80));

pplicationContainer sinkApps = sink.Install (nodes.Get (sinkNode));

sinkApps.Start (Seconds (0.0));

sinkApps.Stop (Seconds (2000.0));

DynamicCast<PacketSink> (sinkApps.Get (0))->TraceConnectWithoutContext ("Rx",           MakeCallback(cb_tcp_sink_received)); 

}

FlowMonitorHelper flowHelper;

Ptr<FlowMonitor> flow_monitor = flowHelper.InstallAll();

    

//----------------------------------------------------------------------------------

//--- SIMULATION

// NetAnim - Create the animation object and configure for specified output

AnimationInterface anim (animFile);  

// Stoppen der Simulation notwendig, da OLSR Packete neue Events generieren

Simulator::Stop (Seconds (2000.0));

NS_LOG_INFO ("Run Simulation.");

std::cout << "Testing from node " << sourceNode << " to " << sinkNode << std::endl; 

std::cout << "----------" << std::endl;

    

Simulator::Run ();

// Flow Monitor

flow_monitor->CheckForLostPackets ();

Ptr<Ipv4FlowClassifier> classifier = DynamicCast<Ipv4FlowClassifier> (flowHelper.GetClassifier ());

std::map<FlowId, FlowMonitor::FlowStats> stats = flow_monitor->GetFlowStats ();

        uint32_t txPacketsum = 0;

        uint32_t rxPacketsum = 0;

        double Delaysum = 0;

        for (std::map<FlowId, FlowMonitor::FlowStats>::const_iterator i = stats.begin (); i != stats.end (); ++i)

        {

Ipv4FlowClassifier::FiveTuple t = classifier->FindFlow (i->first);

if ((t.destinationAddress == Ipv4Address("10.0.0.2")) && (t.sourceAddress == Ipv4Address("10.0.0.1"))){

                txPacketsum += i->second.txPackets;

                rxPacketsum += i->second.rxPackets;

            }

        }

        std::cout << "  All Tx Packets: " << txPacketsum << "\n";

        std::cout << "  All Rx Packets: " << rxPacketsum << "\n";

        std::cout << "  Packets Delivery Ratio: " << ((rxPacketsum *100) / txPacketsum) << "%" << "\n";

        

        flow_monitor->SerializeToXmlFile("jnc_flow", true, true);

Simulator::Destroy ();

}

[Konstantinos]

Hi Jan,

It might be an error in the FlowMonitor classification or could be a misunderstanding of the output.

FlowMonitor will add a probe on any outgoing packet that triggers the Ipv4L3Protocol::SendOutgoing trace source

202  if (!m_ipv4->TraceConnectWithoutContext ("SendOutgoing",

203  MakeCallback (&Ipv4FlowProbe::SendOutgoingLogger, Ptr<Ipv4FlowProbe> (this))))

204  {

205  NS_FATAL_ERROR ("trace fail");

206  }

This callback will try classify the packets that have a valid TCP/UDP header 

https://www.nsnam.org/docs/doxygen/classns3_1_1_ipv4_flow_classifier.html#ac91b7a15892c52b4361de76eaf8411b7

I would recommend to create your own callback on the same trace source (Ipv4L3Protocol::SendOutgoing) and check if you see 5 or 10 TCP packets.

With some smart debugging, you should be able to see in which of the three control checks of Ipv4Classifier::Classify() method those 5 extra packets fail to be classified.

(a) Fragments, (b) non TCP/UDP header and (c) payload < 4.

106  if (ipHeader.GetFragmentOffset () > 0 )

107  {

108  // Ignore fragments: they don't carry a valid L4 header

109  return false;

110  }

111 

...

116 

117  if ((tuple.protocol != UDP_PROT_NUMBER) && (tuple.protocol != TCP_PROT_NUMBER))

118  {

119  return false;

120  }

121 

122  if (ipPayload->GetSize () < 4)

123  {

124  // the packet doesn't carry enough bytes

125  return false;

126  }

[Tommaso Pecorella]

Hi Jan,

indeed it's a bit puzzling. Can you send me the script ? I'll try to check what's the problem, maybe you found a bug (or at least a behaviour worth to be documented).

Cheers,

T.

[Jan H]

Hi Tommaso, 

I sent you an email to your University mail account with the code files. Maybe you have to check your junk mail, because I don't know how your University reacts on attached code.

I could find some additional strange behavior if I disable the MAC retransmission function, the flow monitor will count the TCP packages correct in this case. On the sender and on the reciever side. But if I re-enable this, it is the same behavior as before. At the moment I have no idea why this happens, but I will do some debugging and maybe I will find the mistake.

jan@JTP:~/ns-3-dev$ ./waf --run "scratch/JNC/JNC --op_mode=1 --tracing=true --maxBytes=4000 --mac_retr_off=false"

Waf: Entering directory `/home/jan/ns-3-dev/build'

Waf: Leaving directory `/home/jan/ns-3-dev/build'

Build commands will be stored in build/compile_commands.json

'build' finished successfully (1.157s)

----------

Positions at time t=0s:

    Node #0 has pos 0:0:0

    Node #1 has pos 30:0:0

----------

Testing from node 0 to 1

----------

Time: 30, 3.32, 0.015174

  All Tx Packets: 11

  All Rx Packets: 11

  Packets Delivery Ratio: 100%

jan@JTP:~/ns-3-dev$ ./waf --run "scratch/JNC/JNC --op_mode=1 --tracing=true --maxBytes=4000 --mac_retr_off=true"

Waf: Entering directory `/home/jan/ns-3-dev/build'

Waf: Leaving directory `/home/jan/ns-3-dev/build'

Build commands will be stored in build/compile_commands.json

'build' finished successfully (1.179s)

----------

Positions at time t=0s:

    Node #0 has pos 0:0:0

    Node #1 has pos 30:0:0

----------

Testing from node 0 to 1

----------

Time: 30, 3.32, 26.222304

  All Tx Packets: 22

  All Rx Packets: 13

  Packets Delivery Ratio: 59%

Thanks a lot!

Greetings

Jan

[Konstantinos]

Hi Jan,

I think that the results are as expected.

With MAC re-transmissions, that extra 10 packets are at the MAC layer, hence they are not counted by IP FlowMonitor.

Without re-transmissions, TCP is responsible for those, hence they are seen at the IP from FlowMonitor.

[Jan H]

Konstantinos, I think you're right. I have checked the flags of the IEEE 802.11 frames. And half of them are MAC retransmissions. Stupid of me that I have forgotten. Sorry for the stolen time. And many thanks for the support!

[Konstantinos]

Hi Jan,

No problem, time well spent. 

All of us have gained something from this and future users could use it as reference when explaining their results.

Cheers

K.

[Tommaso Pecorella]

Hi Jan,

my mail was already closed when you sent the files, but it seems that you found the problem. Strange enough, wireshark was showing the packets as retransmissions, but it didn't say that they were MAC-level retransmissions (or I haven't noticed).

Cheers,

T.