SO_RCVBUF and SO_SNDBUF set to 0 inconsistency?

[Pieter Viljoen]

Hi

Until recently I used 0 buffer sizes for my IOCP socket servers. Now I read
an MSDN article claiming otherwise. Well actually that article conflicts
with other MSDN articles. Who am I to believe?

----------
From Q181611: "Socket Overlapped I/O Versus Blocking/Non-blocking Mode":
"If you use the SO_RCVBUF and SO_SNDBUF option to set zero TCP stack receive
and send buffer, you basically instruct the TCP stack to directly perform
I/O using the buffer provided in your I/O call. Therefore, in addition to
the non- blocking advantage of the overlapped socket I/O, the other
advantage is better performance because you save a buffer copy between the
TCP stack buffer and the user buffer for each I/O call."
----------
From MSDN Magazine October 2000 "Windows Sockets 2.0: Write Scalable Winsock
Apps Using Completion Ports" by Anthony Jones and Amol Deshpande:
"Before going apoplectic over the buffer copying that's involved in sending
and receiving data, a programmer should take great care to understand the
consequences of turning off the buffering in AFD.SYS, which can be done by
setting the SO_SNDBUF and SO_RCVBUF values to 0 using the setsockopt API."
... "An even more significant problem with this approach is that your
application can only do one send at a time in each thread. This is extremely
inefficient, to say the least. Turning off receive buffering in AFD.SYS by
setting SO_RCVBUF to 0 offers no real performance gains." ... "It should be
clear by now that turning off buffering is a really bad idea for most
applications. Turning off receive buffering is not usually necessary, as
long as the application takes care to always have a few overlapped WSARecvs
outstanding on a connection."
----------

1. I have always understood that setting buffer sizes to 0 is very good for
IOCP since it eliminates one memory copy, do I believe article 1 (and most
people I talk to) or article 2, or is there a special reason I am missing?

2. From experience, and discussed in this group, the order of read
completions are not guaranteed when using multiple async reads (i.e. async
read 1, async read 2, async read 2 complete, async read 1 complete, stream
corruption). For performance reasons I may want to have multiple reads
outstanding but if the completion order can not be guaranteed that is not
feasible. Is it possible to guarantee completion order?

Regards
Pieter

[Wei Hua]

Hi Pieter,

If you set the SO_RCVBUF to 0, you want to make sure you have multiple
overlapped recv posted. This statement is valid for NT4 and Win2K. But
setting SO_RCVBUF to 0 is no longer necessary on Win2K because Win2K is
smart enough to copy data directly to your recv buffer. The Q181611 was
written in NT4 time frame and is a bit outdated. Anthony and Amol's
article was written after Win2K was shipped.

I hope this helps.

Thanks,
Wei Hua
Microsoft Developer Support

[Wei Hua]

For your question 2, you need to extend the overlapped structure so that
you can sequence them. After the overlapped I/O has completed, even if they
are completed out of order, you can put them back in order based on the
sequenced number in the returned overlapped structure.

Thanks,
-Wei

[Pieter Viljoen]

Wei

Thank you for the reply.

Does that mean that on Win2K if I do not set the SO_RCVBUF size to 0 I do
not have to use multiple outstanding reads? Would that not still require an
internal buffer copy to the user supplied buffer, making a 0 buffer size
with a user supplied buffer better (with the added logic of reordering the
requests)?

Would there be any benefit in changing the default SO_RCVBUF and SO_SNDBUF
size? I already allocate IO buffers using VirtualAlloc() to get them page
aligned, and I round the required size up to an even page boundry (I also
assume the page size will be an even multiple sector size because in some
implementations I actually swap out socket IO and file IO buffers to move
data asynchronously between sockets and files).

Regards

Pieter

"Wei Hua" <weihua_...@microsoft.com> wrote in message
news:eDBJhyW...@cppssbbsa01.microsoft.com...