Still unclear - CompletionPortIO vs. Overlapped IO with Socket class
Roy Chastain
"Please post your questions in the previous message thread; otherwise
the person who replied you in the thread will not know your further
question."
I responded to that post that the 2nd tread was created because the
1st Microsoft employee quite responding to the 1st thread.
That 2nd thread was never responded to again. Now the 3rd thread is
being attempted. My original post on this topic as 3/21/03 and I am
still trying to get a understanding of just what has been implemented
and how it works. This information is important from a scaling point
of view. I firmly believe that GetQueuedCompletionStatus will scale
much better than any of the other async callback and wait mechanisms.
In a previous message thread a Microsoft employee stated the following
"First, Overlapped IO is different from CompltionPort IO. Please
reference MSDN to clarify this two terms."
and then went on to say that the Socket class would use Overlapped IO
when the OS did not support IOCompletionPort IO.
I really would like an explanation of this answer.
1) - Exactly which document would you suggest that I read to "to
clarify this two terms". I really think that I understand Overlapped
IO and Completion Ports. I have several thousand lines of code that
use Completion Ports via the GetQueuedCompletionStatus. One must use
an Overlapped structure to do that.
2) - Given that one must have an Overlapped Structure to do Completion
Port operations, I don't really understand the 2nd part of the answer
that Overlapped IO only happens if Completion Port IO is not
supported. There should not be any cause effect relationship here.
3) - When I run code using Socket class on a Win2K system with
BeginConnect, BeginSend etc and I look at the Socket object with the
debugger, the flag UseOverlappedIO is NOT set. I have also decoded
some of the Socket class code with AnaKrino and, from what I can tell,
if the flag UseOverlappedIO is cleared then the code does not call
WSASend etc with an overlapped pointer etc.
4) - Now the final and most important question really boils down to is
it really possible to get the current framework code to do Overlapped
IO with Completion Ports and a mechanism that is similar to
GetQueueCompletionStatus or at least completion port event
notification (not near so nice).
The reason that this is important to me is that I am trying to
determine if we have all the threads in the thread pool waiting on one
IO each are if they can all be waiting on a pool of IOs. To clarify,
in the case of GetQueuedCompletionStatus, I have all my IOs (a few
hundred sockets) all using the same completion port and I have 4
threads that call GetQueuedCompletionStatus on that completion port
and they process the IO completion etc. This is, I think, very
efficient. If in the model the framework is using, we have a one to
one correspondence such at a thread waits on only one IO, then that
appears to be a very inefficient use of the thread pool. Any words on
this issue would be appreciated.
-------------------------------------------
Roy Chastain
KMSystems, Inc.
Justin Wan[MSFT]
1, I/O completion ports are used with asynchronous I/O. Completion Port is
an associated to one or more file handlers. It belongs to file storage SDK.
Overlapped IO with Socket belongs to platform SDK. They are directly
irrelevant and don't be expected to be used in the same task; though they
are somewhat related to the Overlapped structure. Overlapped socket uses
the WSAOVERLAPPED struct which is designed to be compatible with the
Windows OVERLAPPED structure; Completion ports will use OVERLAPPED
structure.
2, WSASend/WSARecv can be used only when the socket is overlapped one. That
is, WSASocket with WSA_FLAG_OVERLAPPED is called to create socket.
3, WSASend/WSARecv and WSASocket are the Microsoft Windows-Specific
Extension Functions. Besides them, the Windows Sockets specification
includes all the following Berkeley-style socket routines that were part of
the Windows Sockets 1.1 API, including send and recv and socket. These
functions DON'T support overlapped socket.
4, Per your description, you are using BeginConect and BeginSend...., so I
suppose you are using framework class, Socket class. The class is
Berkeley-style socket. In other words, which doesn't support Overlapped IO.
I hope the informatin helps.
Regards,
Justin Wan
Microsoft Partner Online Support
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
::From: Roy Chastain <r...@kmsys.com>
::Subject: Still unclear - CompletionPortIO vs. Overlapped IO with Socket
class
::Date: Mon, 14 Apr 2003 07:56:39 -0400
::Organization: KMSystems, Inc.
::Reply-To: r...@kmsys.com
::Message-ID: <d18l9v8lvo0luvv7m...@4ax.com>
::X-Newsreader: Forte Agent 1.92/32.572
::MIME-Version: 1.0
::Content-Type: text/plain; charset=us-ascii
::Content-Transfer-Encoding: 7bit
::Newsgroups: microsoft.public.dotnet.framework.sdk
::NNTP-Posting-Host: 66.20.246.162
::Lines: 1
::Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
::Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.sdk:6360
::X-Tomcat-NG: microsoft.public.dotnet.framework.sdk
::
::I posted this message before and a 2nd Microsoft employee said
::
Roy Chastain
1) - It appears that most of the response is not relevant to the .net
framework socket class. Okay, I did not say that I was asking about
it, but we are in the .net framework newsgroup.
2) - Basic socket IO with WSASend/WSAReceive does use Overlaped IO and
completion ports. I have code that does so. MS also has examples to
do that.
4) - Now, you state that the framework Socket class does NOT use
Overlapped IO. Below is an answer from Jun Su that says just the
opposite.
| 1) - Does Socket Class use Overlapped IO?
Yes when the OS doesn't support IOCompletionPort.
| 2) - If so, how do I use it.
Just call BeginXXX.
These answer is given in the thread called IOCompletionCallback in
this same newsgroup.
Also look at the thread Overlapped I/O and Socket Class in this
newgroup. Feroze says
On NT platforms, all async socket functions will use Completionports
under the covers. So, you dont have to do anything.
Now to restate the problem.
I have 1 MS person that says the Socket class does Overlapped IO and
but only if the OS does not support Completion Port IO. (This makes
NO sense).
I have 1 MS person that says the Socket class does Completion Port
I/O.
I have 1 MS person that says the Socket class does not do either
Overlapped or Completion Port IO.
The decompiled code has flags for Overlapped IO (I don't see anything
to support Completion Port.) and when I run my test program the flag
for Overlapped IO is not set.
So my question becomes
1) - Is the design spec for Socket Class to support Overlaped /
Completion Port IO but it has not been implemented yet?
2) - Is the code just broken and it is supposed to work
3) - Are the 2 MS people that thing is does support Overlapped /
Completion reading old doc and you know the reality of the situation.
Do you understand my confusion?
Justin Wan[MSFT]
Yes, in my last post, I talked something irrelevant to .net. My intent is
to let you know some background. The winsocket supports five models:
select,WSAAsyncSelect, WSAEventSelect, overlapped I/O and completion port.
(completion port is just supported on Windows 2k and NT).
But this doesn't mean that what an OS supports must be supported on
framework which is installed on the OS, though the future framework might
supports them. In some sense, we can think the functions framework supports
must be subset of functions OS supports.
Please note: completion port model is associated to overlapped I/O model,
because the completion port model requires you to create a Win32 completion
port object that will manage overlapped I/O requests using a specified
number of threads to service the completed overlapped I/O requests.
So, if some platforms or developing tools don't support overlapped I/O
model, the completion port mode should not be supported.
Please check the socket constructor, there is no the parameter with which
we can specify overlapped flag as we can do with the WSASocket API. When we
create a socket without overlapped flag, the socket should not support
overlapped model unless overlapped flag is hard-coded in the implementation
of the socket class. I guess the possiblity is small. This is the reason
you found Overlapped IO was not set when you run your test program. So, I
think overlapped I/O is not supported with the current framework.
I hope I make things clearer.
Regards,
Justin Wan
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
::From: Roy Chastain <r...@kmsys.com>
::Subject: Re: Still unclear - CompletionPortIO vs. Overlapped IO with
Socket class
::Date: Tue, 15 Apr 2003 06:56:45 -0400
::Organization: KMSystems, Inc.
::Reply-To: r...@kmsys.com
::Message-ID: <4don9v0cb6jctk1pa...@4ax.com>
::References: <d18l9v8lvo0luvv7m...@4ax.com>
<rPlMu0yA...@cpmsftngxa06.phx.gbl>
::X-Newsreader: Forte Agent 1.92/32.572
::MIME-Version: 1.0
::Content-Type: text/plain; charset=us-ascii
::Content-Transfer-Encoding: 7bit
::Newsgroups: microsoft.public.dotnet.framework.sdk
::NNTP-Posting-Host: 66.20.246.162
::Lines: 1
::Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP12.phx.gbl
::Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.sdk:6374
::X-Tomcat-NG: microsoft.public.dotnet.framework.sdk
::
::I can't say that this helped a lot.
::