Low TCP Performance Issue with LTE/EPC Model

[Kenneth Zhu]

Hi, all

I'm trying to use ns-3's LTE model to evaluate the application layer performance of TCP.
However, I'm surprised to see that it only got 1.5Mbps as a single pair of UE and eNB is putted together with default configuration.
I think it's too much lower below the theoretical upper bound of 18.6Mbps @5MHz, SISO.

Although most of my code are from the tutorial, I attach them below.
Could any of you tell me where is wrong? Or, what should I do to get a performance close to the theoretical value?

Thank you very much!

Best regards,
Kenneth


------

NodeContainer ueNodes; ueNodes.Create(1);
NodeContainer enbNodes; enbNodes.Create(1);

Ptr<LteHelper> lte = CreateObject<LteHelper> ();
Ptr<PointToPointEpcHelper>  epc = CreateObject<PointToPointEpcHelper> ();
lte->SetEpcHelper (epc);

Ptr<Node> pgw = epc->GetPgwNode ();

NodeContainer servers; servers.Create (1);
Ptr<Node> server = servers.Get (0);
InternetStackHelper internet;
internet.Install (servers);

PointToPointHelper p2p;
p2p.SetDeviceAttribute ("DataRate", DataRateValue (DataRate ("10Gb/s")));
p2p.SetDeviceAttribute ("Mtu", UintegerValue (5000));
p2p.SetChannelAttribute ("Delay", TimeValue (Seconds (0.010)));
NetDeviceContainer iDevs = p2p.Install (pgw, server);
Ipv4AddressHelper ipv4;
ipv4.SetBase ("1.0.0.0", "255.0.0.0");
Ipv4InterfaceContainer ifaces = ipv4.Assign (iDevs);
Ipv4Address serverAddr = ifaces.GetAddress (1);

Ipv4StaticRoutingHelper routing;
Ptr<Ipv4StaticRouting> pdnStaticRouting = routing.GetStaticRouting (server->GetObject<Ipv4> ());
pdnStaticRouting->AddNetworkRouteTo (Ipv4Address ("7.0.0.0"), Ipv4Mask ("255.0.0.0"), 1);

MobilityHelper enbMob;
enbMob.SetMobilityModel("ns3::ConstantPositionMobilityModel");
enbMob.Install(enbNodes);
MobilityHelper ueMob;
ueMob.SetMobilityModel("ns3::ConstantPositionMobilityModel");
ueMob.Install(ueNodes);

internet.Install (ueNodes);
Ipv4InterfaceContainer ueIface;
ueIface = epc->AssignUeIpv4Address (NetDeviceContainer (ueLteDevs));
for (uint32_t u = 0; u < ueNodes.GetN (); ++u)
{
    Ptr<Node> ueNode = ueNodes.Get (u);
    Ptr<Ipv4StaticRouting> ueStaticRouting = routing.GetStaticRouting (ueNode->GetObject<Ipv4> ());
    ueStaticRouting->SetDefaultRoute (epc->GetUeDefaultGatewayAddress (), 1);
}

lte->Attach (ueLteDevs);

BulkSendHelper source ("ns3::TcpSocketFactory", InetSocketAddress (Ipv4Address ("7.0.0.2"), 9));
source.SetAttribute ("MaxBytes", UintegerValue (0));
source.SetAttribute ("SendSize", UintegerValue (100000));
ApplicationContainer sourceApps = source.Install (server);
sourceApps.Start (Seconds (0.0));
sourceApps.Stop (Seconds (60.0));

PacketSinkHelper sink ("ns3::TcpSocketFactory", InetSocketAddress (Ipv4Address::GetAny (), 9));
ApplicationContainer sinkApps = sink.Install (ueNodes.Get (0));
sinkApps.Start (Seconds (0.0));
sinkApps.Stop (Seconds (60.0));

Simulator::Stop(Seconds(duration + 0.001));

Simulator::Run ();

Ptr<PacketSink> sink1 = DynamicCast<PacketSink> (sinkApps.Get (0));
std::cerr << "Total Bytes Received: " << sink1->GetTotalRx () << std::endl;
// Total Bytes Received: 11481656

Simulator::Destroy ();

------

[Nicola Baldo]

Hi Kenneth,

the LTE module has lots of tests, including many on throughput
http://www.nsnam.org/docs/models/html/lte-testing.html
so I suspect that there is either some weird interaction with TCP, or something wrong in your configuration.

To discuss this further, could you please attach MAC, RLC and PDCP stats, and a program that compiles and runs ok?

[Kenneth Zhu]

Hi Nicola,

Thank you very much for your suggestion, I tried to run the test and found nothing special.
In fact I'm working on the ns-3-dev pulled down last Friday without any modification
and my code has no special configuration on TCP or LTE/EPC.

I have attached my code and statistic files.

best regards,
Kenneth

[Nicola Baldo]

After looking at your files, I don't see anything wrong.

In particular, looking at DlMacStats.txt you see that transmissions occur in bursts, e.g., between 59.991s  and 59.994s a burst of PDUs is sent with a MAC throughput of 17.568 Mbit/s.
In the burst, a TB size of 2196 is used, which corresponds to the use of MCS 28 (the maximum) with 24 PRBs (the maximum allowed with a bandwidth of 5MHz when resource allocation type 0 is used (see LTE design docs for why)).

So the peak LTE MAC throughput is consistent.

Now, looking at DlPdcpStats, we see a PDCP throughput around 1.736 Mbit/s, which is around the value you mentioned, and about 10 times lower as the peak MAC throughput. This is clearly due to the fact that only bursts of PDUs are transmitted (as evident from DlMacStats.txt) but the channel is idle for large periods of time. I guess that TCP could be the cause for these bursty transmissions.

[Kenneth Zhu]

Hi Nicola,

Thank you very much for your guide on how to look into the LTE statistics files.
I generate and analyze the packet capture trace on the PGW-server link (because LTE interfaces don't support pcap trace),
and yes the server side is sending a bursty traffic in the same way as you have pointed out.
I attach a TCP digest file including timestamp, ethernet frame length, ip addresses, TCP sequence and acknowledge numbers, tcp segment size.

Well, because TCP is designed to send some amount of data and wait for the acknowledgement before the next ones.
I think it is a correct behavior.
The problem is, why the sending window is limited to some value around 1.5Mbps (in application layer, i.e., about 1.7Mbps in LTE MAC layer).

Let's look at the beginning of the TCP digest file, and you can find that the first duplicated acknowledgement which indicates a packet loss happens around line 114.
I don't know where the packet drop happens but I think it could be no where other than the LTE/EPC modules.
The server begins to send data packet since 3.035s, the packet lost is sent at 3.316s and the duplicated ack is occured at 3.331s.
So the total bytes sent before packet loss happened is roughly 31k bytes, and the average rate is definitely less than 1.5Mbps.
In the following lines you can observe that during the slow-start phase, the packet loss constantly happens until the throughput converges to 1.5Mbps.
(Of course, that also happens after slow-start phase as TCP tries to increase the sending window size from time to time.)

Would you please figure out why packet loss happens under such a low throughput condition?
I think that may be the reason why do we have a low TCP throughput in the experiment.


best regards,
Kenneth

[Nicola Baldo]

It could be due to buffer overflow in the RLC UM buffer. Try to increase ns3::LteRlcUm::MaxTxBufferSize and see if the throughput improves.

[Kenneth Zhu]

Thank you very much, Nicola!

I got a throughput of roughly 12Mbps and I think that would be what I want after I modified the parameter to about 128K.

I gave a try to RLC_AM and found a throughput of 4.9Mbps.
I examined the documentation but finding there is no TxBuffer parameter.
Do you have any suggestion on this?

best regards,
Kenneth

[Nicola Baldo]

When using RLC AM, the throughput might be affected by bug 1757
https://www.nsnam.org/bugzilla/show_bug.cgi?id=1757

see also this thread:
https://groups.google.com/d/msg/ns-3-users/CEfmMX3IRBw/mRVuKDBjvOYJ