winsock2 thread safety
MCD
general about the non-thread safety of winsock. by which i mean,
multiple threads performing sends simultaneously (or recv's for that
matter). one thread performing a read and another performing a write
was deemed acceptable (which makes sense), however. trying to find
more detailed answers on the safety of multiple simultaneous sends, i
found these threads:
http://74.125.47.132/search?q=cache:jIAQpAW70_gJ:www.experts-exchange.com/Programming/Languages/CPP/Q_20086268.html+winsock2+%22thread+safe&cd=1&hl=en&ct=clnk
http://www.gamedev.net/community/forums/topic.asp?topic_id=271179
the two comments that seemed to indicate the contrary are quoted
below:
"Winsock 2 implementations should be completely thread safe.
Simultaneous reads / writes on different threads should succeed, or
fail with WSAEINPROGRESS, depending on the setting of the overlapped
flag when the socket is created. Anyway by default, overlapped sockets
are created; so you don't have to worry about it. Make sure you don't
use NT SP6, if ur on SP6a, you should be ok !"
"The same DLL doesn't get accessed by multiple processes as of the
introduction of Windows 95. Each process gets its own copy of the
writable data segment for the DLL. The "all processes share" model was
the old Win16 model, which is luckily quite dead and buried by
now ;-)"
any comments would be much appreciated.
David Schwartz
> hi, i've read quite a few posts in this group and on the internet in
> general about the non-thread safety of winsock. by which i mean,
> multiple threads performing sends simultaneously (or recv's for that
> matter). one thread performing a read and another performing a write
> was deemed acceptable (which makes sense), however. trying to find
> more detailed answers on the safety of multiple simultaneous sends, i
> found these threads:
You really don't tell us what you're talking about, so it's hard to
give you a useful answer. For example, when you talk about "multiple
simultaneous sends", what do you mean? Are you talking about two
threads sending data on the same TCP socket at the same time? Or are
you talking about different TCP sockets? Or UDP? The answers are
different for all of these keys.
Ask a specific question and you will get a specific answer.
DS
MCD
i didn't say what i was talking about? perhaps, but, you didn't state
whether you were listening.
re: your first question... please don't misquote. i didn't vaguely say
"multiple simultaneous sends" with no context. in my first sentence i
said: "[my interest lies in] the non-thread safety of winsock. by
which i mean,
multiple threads performing sends simultaneously." this isn't vague,
however, yes, for someone who condescends the unspecified, yet
obvious, i would have to have added... "ON THE SAME SOCKET"... in
caps, of course.
re: your second question... of course i'm talking about multiple sends
on the same socket! how else would there be thread contention?! if i
were asking about multiple sends on multiple sockets, the question
would be absurd. was that the idea with these questions? to make my
question sound absurd and vague? please give me even a little bit of
credit. i provided two links where the question was asked in yet two
different ways, if you cared to actually read the question, instead of
trying so hard to make it sound unintelligible.
re: your third and 4th question... as stated, i'm interested in
WINSOCK thread-safety which encompasses BOTH. if you'd care to explain
how thread-safety would be different for a tcp send call vs a udp send
call, i'd be glad to listen.
thank you.
Peter Duniho
> i didn't say what i was talking about? perhaps, but, you didn't state
> whether you were listening.
For someone who is looking for help, you sure seem pretty hostile.
I posted a reply earlier, which doesn't appear to have been delivered.
I'll try and repost it; after all, I went to the trouble to write it, I
might as well (look for it elsewhere in this thread).
But, you should keep in mind that people follow these newsgroups and
answer questions mainly because they actually want to help people. If
they tell you that your question was incomplete and can bear
elaboration, instead of snapping at them, you should consider just
following their advice.
You'll find people are a lot more willing to help you when you don't act
like a jerk.
Pete
Peter Duniho
> [...] one thread performing a read and another performing a write
> was deemed acceptable (which makes sense), [...]
>
> the two comments that seemed to indicate the contrary are quoted
> below:
>
> "Winsock 2 implementations should be completely thread safe.
> Simultaneous reads / writes on different threads should succeed, or
> fail with WSAEINPROGRESS, depending on the setting of the overlapped
> flag when the socket is created. Anyway by default, overlapped sockets
> are created; so you don't have to worry about it. Make sure you don't
> use NT SP6, if ur on SP6a, you should be ok !"
How is the above in contradiction with your previous statement?
> "The same DLL doesn't get accessed by multiple processes as of the
> introduction of Windows 95. Each process gets its own copy of the
> writable data segment for the DLL. The "all processes share" model was
> the old Win16 model, which is luckily quite dead and buried by
> now ;-)"
How is the above related to Winsock at all?
Winsock should in fact be thread-safe. But that doesn't mean that for a
given socket you can just go doing simultaneous sends in multiple
threads, or simultaneous reads in multiple threads willy-nilly.
Simultaneous sends should succeed at the API level (i.e. no error would
be returned), but unless you do some synchronization on the sends, there
is little to ensure that the data being sent is actually in the order in
which you intended.
I suspect, but am not sure of, that if you are using IOCP that at worst
simultaneous sends will result in buffers being delivered out-of-order.
As long as each buffer you send in a given call represents a single,
complete unit of data, it would be fine. But the buffers may be
transmitted in an order other than what was originally expected, and
thus the recipient could receive data in the wrong order.
For non-IOCP sends, there's not even any chance it could work out
reliably. Data sent can be interleaved, sends can be partial, etc.
Similar issues exist for receives. Multiple threads all trying to
receive at the same time may receive different parts of the data stream,
and there's no way to correlate what's been received with the order it
was originally sent. Again, IOCP is better: as long as you post your
buffers in a known order, you're ensured that the buffers will be filled
in the order you posted them. So when the receives actually complete,
you can use that information to preserve order. But you still need
ordering on the actual _posting_ of the buffers (i.e. the call to
WSARecv()) to ensure that you know the order the buffers were posted.
Note that the above all assumes we're talking about a single TCP
connection. A message-based protocol like UDP obviously keeps messages
together, and UDP doesn't guarantee ordering anyway, so there's no
problem with code that also does guarantee ordering during sending or
receiving.
And of course, there should be no problem with thread-safety with
regards to one thread handling one socket and a different thread
handling a completely different socket. The issues come up with you
have more than one thread trying to use a specific socket.
Pete
MCD
wrote:
hi pete,
there's a difference between being hostile and standing up for
yourself. i was doing the latter.
i believe in helping people as well, but i don't make them grovel or
debase themselves for it. it doesn't seem like you do either. enough
said.
thanks for the reply.
MCD
wrote:
> MCD wrote:
> > [...] one thread performing a read and another performing a write
> > was deemed acceptable (which makes sense), [...]
>
> > the two comments that seemed to indicate the contrary are quoted
> > below:
>
> > "Winsock 2 implementations should be completely thread safe.
> > Simultaneous reads / writes on different threads should succeed, or
> > fail with WSAEINPROGRESS, depending on the setting of the overlapped
> > flag when the socket is created. Anyway by default, overlapped sockets
> > are created; so you don't have to worry about it. Make sure you don't
> > use NT SP6, if ur on SP6a, you should be ok !"
>
> How is the above in contradiction with your previous statement?
my quote at the top was the example of what i found in these
newsgroups to be acceptable practice regarding multiple threads on a
single socket. the problem scope i was referring to was the non-
acceptable practice i outlined in the first sentence which was a
socket with multiple threads performing mutliple sends. i believe the
comment i quoted to be in contradiction to that because i interpreted
the author to be stating that it would be impossible for another
thread to make a call on a blocking socket with an active send/recv in
progress and would receive a WSAEINPROGRESS error if attempted.
however, upon doing further research on that error, the definition
from msdn seems to indicate that winsock only allows a single blocking
operation per thread for that error to be given, but it's unclear
whether they meant per thread single socket or per thread multiple
sockets. will most likely need to create a specific test to be
certain.
>
> > "The same DLL doesn't get accessed by multiple processes as of the
> > introduction of Windows 95. Each process gets its own copy of the
> > writable data segment for the DLL. The "all processes share" model was
> > the old Win16 model, which is luckily quite dead and buried by
> > now ;-)"
>
> How is the above related to Winsock at all?
he's referring to the winsock dll, as the question he's answering in
the link i gave asked about winsock thread-safety with the argument
that it should be thread-safe due to it being a dll call.
> Note that the above all assumes we're talking about a single TCP
> connection. A message-based protocol like UDP obviously keeps messages
> together, and UDP doesn't guarantee ordering anyway, so there's no
> problem with code that also does guarantee ordering during sending or
> receiving.
ok, thank you. i think this is what i was looking for. your comments
above weren't necesarily specific to tcp send/recvs themselves, but
rather tcp implementations at the user level. for instance, in your
example, the error in calling recv from multiple threads without being
aware of buffer order is only impractical if buffer order is required.
however, the actual act of simultaneously calling the same socket in
and of itself does not seem in and of itself to cause failure or
undefined behavior. the same i believe would apply to sends... the
packets themselves it seems are not going to be interleaved, however
if your applications require packet ordering you'll be in a heap of
trouble.
i believe some testing is in order on my end. very much appreciate
your response. if you see any glaring misunderstandings on my part,
please let me know.
thank you, mark
Peter Duniho
> there's a difference between being hostile and standing up for
> yourself. i was doing the latter.
No one was putting you in a position where you should feel a need to
stand up for yourself.
> i believe in helping people as well, but i don't make them grovel or
> debase themselves for it.
There was nothing about David's post that in any way suggested you
should grovel or debase yourself.
Pete
Peter Duniho
> [...] i interpreted
> the author to be stating that it would be impossible for another
> thread to make a call on a blocking socket with an active send/recv in
> progress and would receive a WSAEINPROGRESS error if attempted.
> however, upon doing further research on that error, the definition
> from msdn seems to indicate that winsock only allows a single blocking
> operation per thread for that error to be given, but it's unclear
> whether they meant per thread single socket or per thread multiple
> sockets. will most likely need to create a specific test to be
> certain.
I believe it's "per thread", period. At least, that's how I read the
description of the error message.
But I think normally you shouldn't run into that. It would only come up
if a given thread was already blocked on some Winsock API call, and _in
that same thread_ you attempted to make another Winsock API call. That
might occur if you used a callback in one of the Winsock functions, and
that callback happened to get executed in a thread that was already
blocked on a Winsock call (e.g. via an asynchronous procedure call).
But most Winsock code should not run into that.
In other words, my understanding of WSAEINPROGRESS is that it doesn't
have anything to do with Winsock being thread-safe or not. It has to do
with Winsock not being _re-entrant_.
>
>>> "The same DLL doesn't get accessed by multiple processes as of the
>>> introduction of Windows 95. Each process gets its own copy of the
>>> writable data segment for the DLL. The "all processes share" model was
>>> the old Win16 model, which is luckily quite dead and buried by
>>> now ;-)"
>> How is the above related to Winsock at all?
>
> he's referring to the winsock dll, as the question he's answering in
> the link i gave asked about winsock thread-safety with the argument
> that it should be thread-safe due to it being a dll call.
The original question makes a completely irrelevant inference based on
the fact that Winsock has its own DLL. Code existing in a DLL has
nothing at all to do with being thread-safe.
He bases that irrelevant inference on an incorrect belief that multiple
processes not only use the same DLL image in memory, but also share
data. The person who replied is simply pointing out, independent of any
threading question, that DLLs under 32-bit Windows do not share data.
Even in the old model, if data was shared, that does not mean that the
DLL was necessarily written to support a multi-threaded usage. In fact,
lacking any sort of multi-threading support in the early Windows API,
there would have been no reason for a DLL to do so.
The bottom line there is that the question of code being in a DLL or not
has _nothing_ to do with whether that code is thread-safe.
>> Note that the above all assumes we're talking about a single TCP
>> connection. A message-based protocol like UDP obviously keeps messages
>> together, and UDP doesn't guarantee ordering anyway, so there's no
>> problem with code that also does guarantee ordering during sending or
>> receiving.
>
> ok, thank you. i think this is what i was looking for. your comments
> above weren't necesarily specific to tcp send/recvs themselves, but
> rather tcp implementations at the user level. for instance, in your
> example, the error in calling recv from multiple threads without being
> aware of buffer order is only impractical if buffer order is required.
You are correct. "Thread-safe" simply means that the API won't break
(i.e. crash, corrupt data, return unpredictable results, etc.) if you
call it from multiple threads. In that sense, Winsock is thread-safe
and there's no fundamental error in calling recv() from multiple
threads, even for the same socket.
But for a TCP socket, buffer order _is_ required. A given call to
recv() can return as little as a single byte. Unless there is something
about your data that allows you, with even just a single byte, to know
where in the byte stream that byte belongs, you have to have some way of
tracking the correct order of the data in the byte stream.
IOCP provides this for receives by guaranteeing that buffers will be
filled in the order you provide them, even if you call WSARecv()
multiple times before any given receive completes. But you _still_ need
to synchronize those calls, so that _your_ code knows what order the
buffers were provided to Winsock.
> however, the actual act of simultaneously calling the same socket in
> and of itself does not seem in and of itself to cause failure or
> undefined behavior.
Correct. But it still may be useless behavior. :)
> the same i believe would apply to sends... the
> packets themselves it seems are not going to be interleaved, however
> if your applications require packet ordering you'll be in a heap of
> trouble.
If you are thinking about "packets", you had better be talking about a
UDP socket. If you are thinking about "packets" in the context of a TCP
socket, it suggests you've overlooked a very important characteristic of
a TCP socket: it is a stream-oriented object, not message (or
packet)-oriented.
With TCP, you cannot count on a call to recv() to retrieve exactly the
same data that was sent in a single call to send(). You could wind up
getting anywhere from just a single byte, all the way up to the size of
the buffer you provided. This has obvious implications with respect to
your code's ability to reconstruct the correct order of the data if it's
not already aware of the order that calls to recv() or WSARecv() for a
given socket were made and completed, and your code cannot be aware of
that unless it itself is thread-safe (i.e. includes synchronization to
ensure that no two threads are calling recv() or WSARecv() at the same
time).
Pete
MCD
wrote:
i'm sorry pete, but quoting out of context is as useful a tactic in
propaganda as it is in passive-aggressive newsgroup peacocking. it's
meant to cast doubt, to debase, to reduce. period. his repsonse was
akin to me quoting selectively from your reply with, "you should
grovel", and then going on a tirade about this suggestion. i don't
believe david is an idiot, nor do i believe he lacked understanding in
the question. so what was the purpose of those questions then if not
to debase the questioner?
you as well as i know this is common practice in newsgroups... if the
answering participants can find no incentive in answering the question
(eg. to refresh their own knowledge) and there is a level of
insecurity in the individual answering, then it's inevitable that
there will be a requirement of affirmation of status and/or rank. the
result is usually the demanding from the questioner of superfluous
detail or clarification when none is required. when the questioner
accepts this proposition he agrees to this model, he agrees to debase
himself, to elevate the the individual re-questioning with underhanded
remarks, in the hopes that he may be blessed with an answer. forgive
me for not accepting this model. it's tired. it's inefficient. it's
the reason why newsgroups such as these are losing favor to places
like stackoverflow. the last winsock question asked on this group was
the middle of january, and before that the middle of december. replies
like david's do NOT make a community better. questions like mine do
and answers like yours do. however if answer's like david's are
preferenced here, judging from your staunch defense, then i'll leave
you to him.
good luck to you pete.
Peter Duniho
> [...] good luck to you pete.
You too. Looks like you're going to need it.
David Schwartz
> re: your first question... please don't misquote. i didn't vaguely say
> "multiple simultaneous sends" with no context. in my first sentence i
> said: "[my interest lies in] the non-thread safety of winsock. by
> which i mean,
> multiple threads performing sends simultaneously." this isn't vague,
> however, yes, for someone who condescends the unspecified, yet
> obvious, i would have to have added... "ON THE SAME SOCKET"... in
> caps, of course.
> re: your second question... of course i'm talking about multiple sends
> on the same socket! how else would there be thread contention?!
Have you ever worked with a library that actually wasn't thread safe?
If Winsock actually wasn't thread safe, multiple concurrent operations
could cause problems, even on different sockets. For example, when you
initiate a Winsock operation, you pass a socket handle. The table of
valid socket handles and the underlying structures they refer to is a
process-wide shared resource. If Winsock were not thread safe,
multiple concurrent operations, even on different sockets, could cause
problems accessing this shared resource. That's just an example.
Another example is timers. Operating on a TCP connection changes the
state of that TCP connections which may require setting or clearing
timers. Often timers are consolidated into a single internal timer
that fires when the first of the application-level timers needs to
fire. Two concurrent operations, even on different sockets, may wind
up setting two timers at the same time, which could lead to incorrect
operation if the internal timer table wasn't thread safe.
If Winsock weren't thread safe, there could be any number of ways
concurrent operations could fail, the obvious and the subtle.
> if i
> were asking about multiple sends on multiple sockets, the question
> would be absurd.
How would it be absurd? Why isn't it reasonable to ask whether Winsock
is guaranteed thread-safe in that case?
> re: your third and 4th question... as stated, i'm interested in
> WINSOCK thread-safety which encompasses BOTH. if you'd care to explain
> how thread-safety would be different for a tcp send call vs a udp send
> call, i'd be glad to listen.
I don't think you would be glad to listen, but I'll tell you anyway.
With concurrent sends on the same UDP socket, there is no way data can
be intermingled. Each send requests a distinct datagram to be
transmitted. With concurrent TCP sends, nothing "glues" the bytes in
each send together, and intermingling the data between the two sends
together is perfectly reasonable.
It's also hard to imagine a situation where two concurrent TCP writes
to the same socket are useful. If one sends "ab" and the other sends
"cd", even with the strongest thread safety imaginable, it's still
arbitrary whether you send "abcd" or "cdab". Whereas with UDP (for
example, dns) you may not care the order datagrams are sent.
DS