Problem with winsock2 to setup a constant bit rate, please help

[charlie]

Hi,

I am trying to figure out how to set up a high and constant bit rate
(about 100 Mbits/s) of network traffic under Windows XP with winsock2
functions.
I have a simple client/server application running on 2 computers, both
running windows XP, on Intel motherboard DP965LTCK with core 2 duo
processor, and Intel 82566DC gigabit network adapter. Both client and
server are running a simple while loop, to send and receive data via
TCP/IP, to set up a constant rate of network traffic:

///////////////////////////////////////////////////////////////
// CLIENT test constant bit rate = 100 Mbits/s
int port = 33333;
int sleepTime = 10; // sleep for 10 ms between each send
int blockSize = 31250; // send 31250 float (= 1 Mbits) each call
float *bufferToSend = new float[blockSize];
memset(bufferToSend,0,sizeof(float) * blockSize);

.........................................

int bytesSent = 0;
while(1)
{
bytesSent = send(client, (char *)bufferToSend, blockSize *
sizeof(float), 0);
Sleep(sleepTime);
}

///////////////////////////////////////////////////////////////
// SERVER test constant bit rate = 100 Mbits/s
int blockSize = 3*31250; // can receive 3 * 31250 float each call
float * bufferToRcv = new float[blockSize];
memset(bufferToRcv,0,sizeof(float) * blockSize);

...................................................

int bytesRecv = 0;
while(1)
{
bytesRecv = recv(client, (char *)bufferToRcv, blockSize*sizeof(float),
0);
}

According to Windows Task Manager, the applications only require about
2% of the CPU and 10% of the network bandwidth (which is well
100Mbits/s). The problem is that even if the bit rate is well constant
most of the time, big gaps appear in the traffic intermittently. It
seems that the client just stops sending packets for a few 10th of
seconds (or the server stops receiving?) every 15-20 minutes maybe. I
tried with blocking and non-blocking TCP sockets, I tried to connect
the computers with a crossover cable, with a gigabit switch, I also
tried to run this application on different computers with different
network adapters and each time I found the same kind of gaps, more or
less frequently. What's wrong?

Thanks a lot for any help, best regards,

Charles

[Michael K. O'Neill]

First off, don't cross-post. You posted this question at
microsoft.public.win32.programmer.networks and at
comp.os.ms-windows.programmer.networks, and who knows where else too. Learn
how to multi-post instead.

Anyway, see in-line below:

"charlie" <charles...@gmail.com> wrote in message
news:1161232368.3...@i3g2000cwc.googlegroups.com...

> Hi,
>
> I am trying to figure out how to set up a high and constant bit rate
> (about 100 Mbits/s) of network traffic under Windows XP with winsock2
> functions.

100 Mbits/s is *already* the highest limit of most ethernet LANs today. So,
it doesn't make sense that you are somehow trying to limit bandwidth to this
number (e.g., with the use of Sleep() in your code below). You should
instead be trying to do everything to avoid delays, not introduce them.

> I have a simple client/server application running on 2 computers, both
> running windows XP, on Intel motherboard DP965LTCK with core 2 duo
> processor, and Intel 82566DC gigabit network adapter. Both client and
> server are running a simple while loop, to send and receive data via
> TCP/IP, to set up a constant rate of network traffic:
>
> ///////////////////////////////////////////////////////////////
> // CLIENT test constant bit rate = 100 Mbits/s
> int port = 33333;
> int sleepTime = 10; // sleep for 10 ms between each send
> int blockSize = 31250; // send 31250 float (= 1 Mbits) each call
> float *bufferToSend = new float[blockSize];
> memset(bufferToSend,0,sizeof(float) * blockSize);
>
> .........................................
>
> int bytesSent = 0;
> while(1)
> {
> bytesSent = send(client, (char *)bufferToSend, blockSize *
> sizeof(float), 0);
> Sleep(sleepTime);

Eliminate the call to Sleep() completely. Don't even call Sleep( 0 ), since
this will cause a thread switch (check the documentation to confirm for
yourself). A thread switch will necessarily introduce significant delays,
corresponding to the time it takes before your thread regains a processor
time-slice.

> }
>
> ///////////////////////////////////////////////////////////////
> // SERVER test constant bit rate = 100 Mbits/s
> int blockSize = 3*31250; // can receive 3 * 31250 float each call
> float * bufferToRcv = new float[blockSize];
> memset(bufferToRcv,0,sizeof(float) * blockSize);
>
> ...................................................
>
> int bytesRecv = 0;
> while(1)
> {
> bytesRecv = recv(client, (char *)bufferToRcv, blockSize*sizeof(float),
> 0);
> }
>
> According to Windows Task Manager, the applications only require about
> 2% of the CPU and 10% of the network bandwidth (which is well
> 100Mbits/s). The problem is that even if the bit rate is well constant
> most of the time, big gaps appear in the traffic intermittently. It
> seems that the client just stops sending packets for a few 10th of
> seconds (or the server stops receiving?) every 15-20 minutes maybe. I
> tried with blocking and non-blocking TCP sockets, I tried to connect
> the computers with a crossover cable, with a gigabit switch, I also
> tried to run this application on different computers with different
> network adapters and each time I found the same kind of gaps, more or
> less frequently. What's wrong?
>
> Thanks a lot for any help, best regards,
>
> Charles
>

Arkady (over in the other NG) has suggested that you play with the sizes of
the socket's buffers. That might work, since to get decent speeds on
Winsock, it's sometimes necessary to change the per-socket buffer sizes
(setsockopt() with SO_SNDBUF and then with SO_RCVBUF) to values that are
larger than the default values.

I don't think that's the problem in your case, however. Here's why:

The default size for buffers in most recent versions of Winsock is 8k for
each of the send and receive buffers (the default setting is found in the
registry).

The "correct" size is determined by the round-trip-time (RTT) of the network
and the bandwidth you are designing for, according to the equation: buffer
size = RTT * bandwidth. High latency networks have a high RTT, and the
buffer must be larger to accommodate this, thereby allowing for more data
"in flight" before transmission pauses to await an ACK.

Your bandwidth target is 100,000,000 bps. For ethernet over a completely
internal LAN, 0.5 msecs is a high (i.e., conservative) estimate of RTT.
Altogether, this yields a buffer size of 0.0005 secs * 100,000,000 bps / 8
bits/byte = 6250 bytes, which is well within the 8k default value.

If you were going out over a network with a considerably higher RTT, then
increasing the buffer size would help. You can "ping" the recipient to get
an idea of latency and RTT for your network.

But assuming RTT is around 0.5 msec, then almost certainly the thing that's
holding down your throughput is the call to Sleep(). Eliminate it, or at
least get a get a good algorithm for adjusting the sleep time to a value
that changes based on the current throughput you are seeing, and doesn't
call Sleep() at all if it would be called with a sleep time of zero (to
avoid the thread switch).

Mike

[Michael K. O'Neill]

"charlie" <charles...@gmail.com> wrote in message

news:1161232708....@f16g2000cwb.googlegroups.com...
> Hi,
>
> I am trying to figure out how to set up a constant bit rate (about 100

> Mbits/s) of network traffic under Windows XP with winsock2 functions.

> I have a simple client/server application running on 2 computers, both
> running windows XP, on Intel motherboard DP965LTCK with core 2 duo
> processor, and Intel 82566DC gigabit network adapter. Both client and
> server are running a simple while loop, to send and receive data via
> TCP/IP, to set up a constant rate of network traffic:
>

> < snip >

So you posted here too.

See my answer on comp.os.ms-windows.programmer.networks

[charlie]

Thanks a lot for your replies, and sorry for the cross post.
The code I sent (with Sleep()) was actually a bad "model" for what
I am trying to do : steaming audio with low latency via TCP.
In fact, the sending loop is done at interrupt level (via the software
Max/MSP). So the send() function is call every 5 ms to send a 65540
Bytes segment. I found out that sometimes send() introduces too much
latency and freeze the sending loop. I tried to increase the SO_SNDBUF
size to several segments (I tried up to 6MB so 10 segments) so that a
send() should always return instantaneously without freezing the
sending loop (I don't want to use a non blocking socket to do that
cause the data has to be sent anyway). Anyway it doesn't work: every
10 minutes, the sending loop is blocked, and the audio stream is
broken.
Can you please tell me if you see something I could change to get it
working?

Thanks a lot, best,

Charles

[Peter Duniho]

"charlie" <charles...@gmail.com> wrote in message

news:1161931449.2...@b28g2000cwb.googlegroups.com...
> [...]

> Can you please tell me if you see something I could change to get it
> working?

I already did (to get the data throttling working, that is).

Beyond that, you cannot rely on TCP/IP over the usual circuits (eg LAN,
Internet) for low-latency, interruption-free transmissions. It's just not
built into the protocols.

In order to avoid interruptions in the data, you need to buffer, which of
course voids any hope for low-latency. You can't have both.

[Michael K. O'Neill]

"charlie" <charles...@gmail.com> wrote in message

news:1161931449.2...@b28g2000cwb.googlegroups.com...

> Thanks a lot for your replies, and sorry for the cross post.
> The code I sent (with Sleep()) was actually a bad "model" for what
> I am trying to do : steaming audio with low latency via TCP.
> In fact, the sending loop is done at interrupt level (via the software
> Max/MSP). So the send() function is call every 5 ms to send a 65540
> Bytes segment. I found out that sometimes send() introduces too much
> latency and freeze the sending loop. I tried to increase the SO_SNDBUF
> size to several segments (I tried up to 6MB so 10 segments) so that a
> send() should always return instantaneously without freezing the
> sending loop (I don't want to use a non blocking socket to do that
> cause the data has to be sent anyway). Anyway it doesn't work: every
> 10 minutes, the sending loop is blocked, and the audio stream is
> broken.
> Can you please tell me if you see something I could change to get it
> working?
>
> Thanks a lot, best,
>
> Charles
>

You could also tr to increase the receiver's buffer, with SO_RCVBUF. The
sliding window is a cooperative effort between sender and receiver, so
increasing only one side often does not have any effect.

However, as pointed out in my calculations (above), I don't think buffer
size is related to the issue you are seeing, unless you have a high-latency
network.

Are you running a gigabit Ethernet? Your transmision rate (65540*8*200 per
sec = 104 Mbits/sec) would exceed the capacity of a standard LAN.

Mike

[William Stacey [C# MVP]]

| First off, don't cross-post. You posted this question at
| microsoft.public.win32.programmer.networks and at
| comp.os.ms-windows.programmer.networks, and who knows where else too.
Learn
| how to multi-post instead.

More people cry about multi-posts IIRC. A cross-post is preferred to
multi-post for a limited number of related groups. IMO, his post was fine.

| Eliminate the call to Sleep() completely. Don't even call Sleep( 0 ),
since
| this will cause a thread switch (check the documentation to confirm for
| yourself).

Sleep(0) may not switch at all depending on the priority of other threads.
If there is not higher priority thread, it may not switch.
Sleep(1) should switch.

--
William Stacey [C# MVP]