Max Size of Socket::SendTo()

[Harald Ott]

I have a client and a server. The client requests a certain amount of bytes that he wants the server to send him.
Both have their callbacks set correctly

socket->SetRecvCallback (MakeCallback (&TcpStreamServer::HandleRead, this));

and everything works just fine, just until the client starts requesting package sizes > 132000 bytes (I haven' figured out the exact 'border', but its around that value).

The server works fine with this code (for package sizes < 132000 bytes)

void
TcpStreamServer::HandleRead (Ptr<Socket> socket)
{
 NS_LOG_FUNCTION (this << socket);

 int result;
 uint32_t packet_size_to_return;
 Ptr<Packet> packet;
 Address from;
 while ((packet = socket->RecvFrom (from)))
 {
 packet_size_to_return = GetCommand(packet);
 result = socket->SendTo (0, m_maxPacketSize, 0, from);
 if (result < 0)
 {
 NS_LOG_ERROR("SendTo of server failed!");
 }
 }
}

And the client works like this:

void
TcpStreamClient::HandleRead (Ptr<Socket> socket)
{

 NS_LOG_FUNCTION (this << socket);
 Ptr<Packet> packet;
 Address from;
 if (m_bytesReceived == 0)
 {
 m_transmissionStartReceivingSegment = Simulator::Now ().GetMicroSeconds () - m_simulationStart;
 if (normal_messages) NS_LOG_INFO ("Transmission Start: " << m_transmissionStartReceivingSegment);
 }
 uint32_t packetSize;
 packet = socket->RecvFrom (from);
 packetSize = packet->GetSize ();
 NS_LOG_INFO ("At time " << Simulator::Now ().GetMicroSeconds () - m_simulationStart << "s client received " << packetSize);
 throughput_log << (Simulator::Now ().GetMicroSeconds () - m_simulationStart) / (double) 1000000 << " " << packetSize << "\n";

 m_bytesReceived += packetSize;

 if (m_bytesReceived == m_segments.at (m_currentRepresentationIndex).at (m_segmentCounter))
 {
 SegmentFullyReceivedHandle();
 }
}

So when I send packets larger than 536 bytes, the socket somehow automatically chops it up into 536 bytes pieces, so the output from the client looks like this:

At time 3018446s client received 536
At time 3020334s client received 536
At time 3022222s client received 536

and so on.

Now I've tried to extend the server so that when I have to send more than 132000 bytes he automatically 'chops' the data in multiple SendTo()s so that I never send more than 132000 via a single SendTo.
I've tried this, but unfortunately after the first successful SendTo(), the following SendTo()s fail, because, looking at the simulation time, no virtual time seems to pass between the multiple SendTo()s but the server would have to wait between them.
Now, I could send another request from the client to send the next "part" after he finished receiving the first, but wouldn't it somehow be possible to do this more elegant?

[Konstantinos]

Hi Harald,

There is something called MTU and the default value for TCP is 536

SegmentSize: TCP maximum segment size in bytes (may be adjusted based on MTU discovery)

• Set with class: ns3::UintegerValue
• Underlying type: uint32_t 0:4294967295
• Initial value: 536
• Flags: construct write read

Regards,

K

[Harald Ott]

I probably should've phrased my question more compactly:
I am already aware of the fact that MTU for TCP is 536.
But when I call SendTo() with more than ~132000 bytes, I get an error.
With < 132000 bytes the framework seems to be able to chop it into 536 byte units.
Why?


[Nat P]

And what is the error? How we could know?

[Harald Ott]

If I call SendTo(), and try to send more than ~132000 bytes it returns -1, meaning that the data could not be sent.

[Nat P]

Il giorno martedì 23 agosto 2016 21:37:14 UTC+2, Harald Ott ha scritto:

If I call SendTo(), and try to send more than ~132000 bytes it returns -1, meaning that the data could not be sent.

Ok, have you tried to look in the code, why there is -1 as return code? An hint: look into TcpSocketBase, that reimplements SendTo, and look into the source code. If you have a debugger, you can easily see what is happening inside the method. It's really easy, I (and many others) can give you the solution right now, but I'd like that you spend some time into debugging it :)

Keep us informed

Nat

[Harald Ott]

I'm having real trouble with this (a few days already), don't think I haven't tried debugging... The debugger doesn't give me the reason why the SentTo() failed, it just shows me what functions are called after the failed SendTo(), namely StopApplication(), HandlePeerClose() and DoDispose().
Please tell me what you think is the problem, if you're sure you know it...
Is it that SendTo() just can't send more bytes? But this would be in conflict with how TCP works, wouldn't it? With TCP I should be able to "push" as many bytes as I want into my socket and behind the scenes it should be split in as many packets as necessary...

[Nat P]

Il giorno mercoledì 24 agosto 2016 14:28:34 UTC+2, Harald Ott ha scritto:

I'm having real trouble with this (a few days already), don't think I haven't tried debugging... The debugger doesn't give me the reason why the SentTo() failed, it just shows me what functions are called after the failed SendTo(), namely StopApplication(), HandlePeerClose() and DoDispose().

It's because you need to stop ("break into" using gdb terminology) inside that function, and precisely at line 814 (https://github.com/kronat/ns-3-dev-git/blob/master/src/internet/model/tcp-socket-base.cc#L814)
 

Please tell me what you think is the problem, if you're sure you know it...

You tried few days, what could be one hour more?
 

Is it that SendTo() just can't send more bytes? But this would be in conflict with how TCP works, wouldn't it? With TCP I should be able to "push" as many bytes as I want into my socket and behind the scenes it should be split in as many packets as necessary...

You're right, but in pratice there is a limit (as always.. theory is theory, which is different from practice :D). Go into that function and look, by stepping one line at time, why you have this problem. Or, you can check the errno status with GetErrno () over the socket (but the debugger way in this case is waaay more fast..).

I'm sorry, I'm not responsive as I would wish... so please, don't look down, or look it after you tried to debug












































(Increase TCP buffer size, that quantity exceeds the default value)




Nat

[Harald Ott]

Thanks for your help Nat P, but I've decided that it wouldn't be good for me to just increase this size, but to solve this problem with a callback function, namely (from Socket class:)

SetSendCallback   (   Callback< void, Ptr< Socket >, uint32_t >   sendCb  )

So I've written my function that handles sending not more than maximum packet size at a time and be called by the callback function again when there's more to send, but I don't know how to fill it in the SetSendCallback function correctly.

m_socket->SetSendCallback (Callback< void, Ptr< Socket >, &TcpStreamServer::HandleSend);

Compiler says template argument 3 is invalid... how do I do this correctly?

[Natale Patriciello]

On 29/08/16 at 06:59am, Harald Ott wrote:
> Thanks for your help Nat P, but I've decided that it wouldn't be good for
> me to just increase this size, but to solve this problem with a callback
> function, namely (from Socket class:)
> SetSendCallback ( Callback< void, Ptr< Socket >, uint32_t > sendCb )

Sorry but this is no-sense, or you should explain why you want to use a
callback. The standard way to handle this is very simple: your TCP buffers holds
~130 kB, if you want to pass (at the same time) more than 130 kB, you
should increase the buffer size. You have a very easy Attribute to do
that. No code edit, no need to worry about nothing.

If you don't understand that, or if you failed to discover this, we need
to update the documentation. Please, if this solve your issue, share
the documentation that you've read and why this was not clear from the
beginning.

Nat

[Harald Ott]

Well, the maximum BufSize would be variable, depending on the input data, I could end up with test data containing single packet sizes > 500.000 kBit.

So, ok, I could just get the largest occuring size in my input data and just set the SndBufSize and RcvBufSize to that value...

But again here I'm having another problem, using Config::Set to change the attributes SndBufSize and RcvBufSize:

I'm using TcpSocketFactory to create my sockets, and I'm trying to set it like this

Config::Set("/NodeList/*/$ns3::TcpL4Protocol/SocketList/*/SndBufSize" , UintegerValue (1000000));

Config::Set("/NodeList/*/$ns3::TcpL4Protocol/SocketList/*/RcvBufSize" , UintegerValue (1000000));

but nothing happens.... the size seems to remain at the default value... what's wrong here ?

[Natale Patriciello]

On 31/08/16 at 08:29am, Harald Ott wrote:
> So, ok, I could just get the largest occuring size in my input data and
> just set the SndBufSize and RcvBufSize to that value...

If you have losses this approach fails as well, because the buffer will
not be empty when you push the next chunk of data. The general way (and
I mean real world) way to deal with this situation is just queueing that
data in the application and send it down at the next notification, which
in ns-3 is the callback "NotifySend". Take a look at the documentation:

https://www.nsnam.org/docs/models/html/tcp.html#tcp-socket-interaction-and-interface-with-application-layer

> But again here I'm having another problem, using Config::Set to change the
> attributes SndBufSize and RcvBufSize:
>
> I'm using TcpSocketFactory to create my sockets, and I'm trying to set it
> like this
> Config::Set("/NodeList/*/$ns3::TcpL4Protocol/SocketList/*/SndBufSize" ,
> UintegerValue (1000000));
> Config::Set("/NodeList/*/$ns3::TcpL4Protocol/SocketList/*/RcvBufSize" ,
> UintegerValue (1000000));
>
> but nothing happens.... the size seems to remain at the default value...
> what's wrong here ?
>

You probably do it before the Socket creation, and so they don't exists in the
moment you set the Config value. They are created in the application at
the starting time, if I remember correctly.

There is a general dis-agreement on how we can improve the situation. My
current proposal is to add a "SetSocket" API on the applications, create
the sockets in the main () function, updating all the attributes on the
real pointers, and then call explicitly

application->SetSocket(mySocket);

basically because I like raw pointers. There are many other variants
(e.g. set a callback that fires 1 ns after the starting time and
configure appropriately with paths), you can use the method you like
more.

PS: I don't like being pushed privately.

Nat

[Harald Ott]

Thanks for your answer, I've set the value too early, like you said,

PS: I'm truly sorry about that. It will not happen again.