tcp receive too slow

[kristof.va...@gmail.com]

Hi All,

I noticed that the download speed to our devices over SMB was not good
enough. The download speed seems to be limited to 5MByte/s, thereby
causeing a lot of CPU activity (up to 100% on a Intel Celeron M
1.5Ghz), on a Intel 85421 1000BaseT ethernet adapter. Uploading data
seems to be working fine though.

At first I suspected the problem was related to the device driver,
that might not be using DMA.
I swapped the ethernet adapter to RTL8169S chipset, but the network
speed did not improve.
I also used the NDIS performance test, and achieved good results
(624Mb/s) receiving data.

perf_ndis Msg: Send 100000 packets : size 1514 bytes : pool 64
packets : 1797 ms : 144.4 MB : 642.8 Mbps : 55648.3 pps : 2.9 idle
perf_ndis Msg: Recv 100000 packets : size 1514 bytes : pool 64
packets : 1798 ms : 144.4 MB : 642.4 Mbps : 55617.4 pps : 65.1
idle : 0.0

So I think the problem is not driver related.

I took a next step by running the Winsock performance test of the
cetk.
This test seems to confirm the problem.
TCP send speed seems to be ok, but receiving data is way too slow.

*** vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
BEGIN TEST: "TCP Recv Throughput", Threads=1, Seed=4316
Recv Packet Bytes Recvd Recv Rate CPU Util
(Bytes) (Bytes) (Kbps) (%)
----------- ----------- ----------- --------
* 16 160000 12307.69 51.92
* 32 320000 36056.34 78.87
* 64 640000 60952.38 95.24
* 128 1280000 63209.88 83.95
* 256 2560000 63209.88 80.86
* 512 5120000 62822.09 78.07
* 1024 10240000 70378.01 80.67
* 1460 14600000 76339.87 84.90
* 2048 20480000 66251.52 73.96
* 2920 29200000 79025.71 84.07
* 4096 40960000 76524.99 81.92
* 8192 81920000 85825.04 90.66
END TEST: "TCP Recv Throughput", PASSED, Time=21.649
*** ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

*** vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
BEGIN TEST: "TCP Send Throughput", Threads=1, Seed=26529
Send Packet Bytes Sent Send Rate CPU Util
(Bytes) (Bytes) (Kbps) (%)
----------- ----------- ----------- --------
* 16 160000 16842.11 98.68
* 32 320000 39384.62 100.00
* 64 640000 45714.29 97.32
* 128 1280000 88275.86 98.28
* 256 2560000 139319.73 78.23
* 512 5120000 219037.43 71.66
* 1024 10240000 286433.57 70.98
* 1460 14600000 412720.85 77.74
* 2048 20480000 340623.70 68.61
* 2920 29200000 350224.89 71.96
* 4096 40960000 343120.42 62.30
* 8192 81920000 347671.09 62.86
END TEST: "TCP Send Throughput", PASSED, Time=5.473
*** ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

running a test with ttcp confirms this behavior.

Now I am a little bit stuck. I believe that my TCP/IP settings are
fairly unchanged from the defaults, and playing a little bit with them
did not really help.

So I was hoping that somebody here might give me a hint on what to do
next, or better, knows the reason for this behavior. Anybody?
Are you seeing the same the CPU utilization when doing network
transfer?
What speeds do you achieve using the winsock performance test on a
1000BaseT ethernet interface?

Any help appreciated,

Kristof Vandenbussche

[Paul G. Tobey [ eMVP ]]

The CPU utilization is irrelevant. All that means is that NDIS and/or the
driver is doing everything it can to perform the operation (you aren't
sitting idle half the time just for the sake of making the CPU utilization be
50%).

As for the speed, that sort of thing is often, though not always, a result
of the packet size and/or receive data pattern employed by the network stack.
You can imagine that, if your TCP/IP test program does recv( socket, 1, ...
) for, say, 1 million times to receive 1MB of data, that's going to be a lot
slower than recv( socket, 1000000, ... ). Obviously, receiving 1MB in a
single recv call isn't going to be practical, but SMB may not be tuned very
well for performance in CE.

Further, you are, in the case of receive, writing this data somewhere, which
is almost always slower than reading it. So, let's say it's flash that
you're reading/writing. Flash writing might be 100 times slower than flash
reading. I guess that you could identify whether this is the source of the
problem by reading/writing your files to RAM disk, where read and write take
the same amount of time.

Of course, you should install all of the QFEs for whatever version of
Windows CE you are using...

Paul T.

"kristof.va...@gmail.com" wrote:

> .
>

[kristof.va...@gmail.com]

Hi Paul,

Thanks for replying to my message.
But I do not agree entirely with your statement that CPU utilization
is irrelevant. Just compare XP and CE on the same X86 hardware. The
difference is huge.
Copying a large file over ethernet places a heavy burden on the CE
system at very slow speeds, while on XP the action is barely
noticeable and lightning fast.

But I do agree that the SMB server - and the receiving medium, which
is a 320GB PATA Hard Disk drive - also need some CPU power and should
be looked at to see if they're working fast enough.
For now, I have identified the network connection itself to be the
bottleneck.

In the meantime I was pointed to Antonie Van Woerdekom's message:
PERFORMANCE enhancement for Ethernet in CE 5.0 and CE 6.0 on x86
(
http://groups.google.be/group/microsoft.public.windowsce.platbuilder/browse_thread/thread/c0fe825420c5483f/9866e2785f51f32f?hl=nl&ie=UTF-8&q=antonie+van+woerdekom+ethernet+fix#9866e2785f51f32f
)
which seems to be promising. It just doesn't explain why the ndis test
is fast and the tcp test slow.
Since I do not have the Intel or Realtek driver source code, I can't
test this easily. But as I can't find any QFE for it, I will take my
question straight to Microsoft.

Thanks everybody,

Kristof Vandenbussche

[Paul G. Tobey [ eMVP ]]

So what? Are you saving those cycles for something? All that matters is
that your real-time tasks happen in the amount of real-time that is suitable.
If it's not, then you may need to do some tuning, either to the performance
of your drivers or your kernel code, or to the priorities of various
real-time threads in your system. The number that you get back as processor
utilization doesn't indicate anything about the badness or goodness of your
system; at best, it's interesting information and nothing more.

Paul T.

"kristof.va...@gmail.com" wrote:

> .
>