Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Is socket thread-safe?

2,472 views
Skip to first unread message

alex

unread,
Nov 3, 2002, 8:35:00 PM11/3/02
to
If a socket is shared between two threads, is it thread if both said threads
send() data via such socket? How about recv()? While one thread gets part
of data in socket buffer sometimes and at other time the other thread will
consume part of the data stream?

Thanks in advance!


Eugene Mayevski

unread,
Nov 4, 2002, 1:46:19 AM11/4/02
to

Sockets are not thread-safe in Windows and if you ever try to do
anything with the socket from another thread, you will get mysterious
errors in Winsock later.

Sincerely yours,
Eugene Mayevski

Sujeet Varakhedi

unread,
Nov 4, 2002, 2:51:21 AM11/4/02
to
Hi! Eugene,
If u say that sockets are not thread safe in windows,the how are they thrad
safe in UNIX or Linux....just wanted to know

Sujeet
"Eugene Mayevski" <maye...@eldos.org> wrote in message
news:aq554s$2ssb$1...@news.dg.net.ua...

those who know me have no need of my name

unread,
Nov 4, 2002, 5:25:30 AM11/4/02
to
in comp.protocols.tcp-ip i read:

>If u say that sockets are not thread safe in windows,the how are they thrad
>safe in UNIX or Linux....just wanted to know

they aren't thread-safe anywhere. you need to serialize access yourself.

--
bringing you boring signatures for 17 years

Eugene Mayevski

unread,
Nov 4, 2002, 9:22:13 AM11/4/02
to
Sujeet Varakhedi wrote:

> If u say that sockets are not thread safe in windows,the how are they thrad
> safe in UNIX or Linux....just wanted to know

I think this is platform- and implementation-specific. On Windows we
tried to close the socket from other thread and this caused large
problems. So I know it's not thread-safe :)

Sincerely yours,
Eugene Mayevski

Phil Frisbie, Jr.

unread,
Nov 4, 2002, 12:42:02 PM11/4/02
to

That sounds more like bad program architecture :)

It is common to close a socket on one thread when there is another thread
blocking on a socket call. When the socket is closed, the blocking call will
unblock and set the error ENOTSOCK. Your thread should recognize that as a
signal to clean up and exit.


Phil Frisbie, Jr.
Hawk Software
http://www.hawksoft.com

Phil Frisbie, Jr.

unread,
Nov 4, 2002, 12:45:21 PM11/4/02
to

It makes no sense to use multiple threads on the same TCP socket sending or
receiving. UDP maybe, but never TCP unless you are using some sort of message
boundaries and mutexes to block the other threads.

However, it IS thread safe, and common, to use one thread to send and another to
receive.

David Schwartz

unread,
Nov 4, 2002, 1:02:17 PM11/4/02
to

They are guaranteed thread-safe in Winsock2, provided you do things
that make sense. For UDP, you can perfectly legally send and receive
from any combination of threads. For TCP, you can send in one thread
while you receive in another.

DS

Casper H.S. Dik

unread,
Nov 4, 2002, 3:25:02 PM11/4/02
to
those who know me have no need of my name <not-a-rea...@usa.net> writes:

>in comp.protocols.tcp-ip i read:

>>If u say that sockets are not thread safe in windows,the how are they thrad
>>safe in UNIX or Linux....just wanted to know

>they aren't thread-safe anywhere. you need to serialize access yourself.

They're thread-safe in Solaris.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

those who know me have no need of my name

unread,
Nov 4, 2002, 4:04:05 PM11/4/02
to
in comp.protocols.tcp-ip i read:

>those who know me have no need of my name <not-a-rea...@usa.net>
>writes:
>>in comp.protocols.tcp-ip i read:

>>>If u say that sockets are not thread safe in windows,the how are they
>>>thrad safe in UNIX or Linux....just wanted to know
>
>>they aren't thread-safe anywhere. you need to serialize access yourself.
>
>They're thread-safe in Solaris.

by themselves they are safe on every platform i know of, but the
programmers invariably do something stupid that transforms them
into uselessness.

Eugene Mayevski

unread,
Nov 4, 2002, 4:18:24 PM11/4/02
to
Phil Frisbie, Jr. wrote:

>>I think this is platform- and implementation-specific. On Windows we
>>tried to close the socket from other thread and this caused large
>>problems. So I know it's not thread-safe :)
> That sounds more like bad program architecture :)
> It is common to close a socket on one thread when there is another thread

Ok, how would one explain that Winsock crashes when the socket is closed
in another thread from where it was created? There is no "program
architecture" at all. Just one socket transferring the data and another
which closes the socket.

I am not trying to defned my point of wview, but rather trying to find
out the truth. We had terible Aaccess violations and other strange
things after doing what I've described.

Sincerely yours,
Eugene Mayevski

David Schwartz

unread,
Nov 4, 2002, 4:29:16 PM11/4/02
to
Eugene Mayevski wrote:

> Ok, how would one explain that Winsock crashes when the socket is closed
> in another thread from where it was created? There is no "program
> architecture" at all. Just one socket transferring the data and another
> which closes the socket.

1) Were you using Winsock2?

2) Are you sure no other thread could possibly have been using that
socket when you closed it?

If your answer to both questions is "yes", you are seeing something
nobody else has reported. If your answer to either question is "no", why
not?!

DS

Phil Frisbie, Jr.

unread,
Nov 4, 2002, 7:38:12 PM11/4/02
to
Eugene Mayevski wrote:
>
> Phil Frisbie, Jr. wrote:
>
> >>I think this is platform- and implementation-specific. On Windows we
> >>tried to close the socket from other thread and this caused large
> >>problems. So I know it's not thread-safe :)
> > That sounds more like bad program architecture :)
> > It is common to close a socket on one thread when there is another thread
>
> Ok, how would one explain that Winsock crashes when the socket is closed
> in another thread from where it was created?

What was the thread doing? Blocking on a Winsock call or something else?

> There is no "program
> architecture" at all.

ALL programs have an architecture; it is simply how you coded it.

> Just one socket transferring the data and another
> which closes the socket.
>
> I am not trying to defned my point of wview, but rather trying to find
> out the truth. We had terible Aaccess violations and other strange
> things after doing what I've described.

Can you code up a single source code sample that shows the problem? Feel free to
post it here or email it directly to me to look at.

> Sincerely yours,
> Eugene Mayevski

alex

unread,
Nov 5, 2002, 1:38:00 AM11/5/02
to
David Schwartz <dav...@webmaster.com> wrote in message
news:3DC6B629...@webmaster.com...

If "They are guaranteed thread-safe in Winsock2", how about Winsock below 2?
Will Winsock1 be thread-safe?

In Winsock2, will send and receive in two thread be a problem?

Thanks for your help!


Alun Jones

unread,
Nov 5, 2002, 6:44:42 AM11/5/02
to
In article <3dc6d79e$0$46603$e4fe...@news.xs4all.nl>, Casper H.S. Dik
<Caspe...@Sun.COM> wrote:
>those who know me have no need of my name <not-a-rea...@usa.net> writes:
>>in comp.protocols.tcp-ip i read:
>
>>>If u say that sockets are not thread safe in windows,the how are they thrad
>>>safe in UNIX or Linux....just wanted to know
>
>>they aren't thread-safe anywhere. you need to serialize access yourself.
>
>They're thread-safe in Solaris.

"thread-safe" doesn't necessarily mean anything, and at the very least it
depends on what you're talking about.

There isn't a "thread-safe" implementation of sockets available on _any_
platform if what you're talking about is two threads sending TCP data on one
socket over the same brief time period. And, whether there was a proscription
against it or not, no sockets implementation is going to give you trouble if
you are using one thread to send and one thread to receive, otherwise that
implementation would be unusable for many applications.

So, yes, sockets are going to be thread safe for sane operations. No, sockets
can't make up for a lack of synchronisation in your own code, if the protocol
requires it (as TCP does). And those two statements will apply to every
sockets implementation - Windows, Unix, Linux, whatever. [Windows 3.1,
obviously, since it doesn't have threads, doesn't have to be thread safe]

Alun.
~~~~

[Please don't email posters, if a Usenet response is appropriate.]
--
Texas Imperial Software | Try WFTPD, the Windows FTP Server. Find us at
1602 Harvest Moon Place | http://www.wftpd.com or email al...@texis.com
Cedar Park TX 78613-1419 | VISA/MC accepted. NT-based sites, be sure to
Fax/Voice +1(512)258-9858 | read details of WFTPD Pro for XP/2000/NT.

Alun Jones

unread,
Nov 5, 2002, 6:44:43 AM11/5/02
to
In article <aq6o8f$1cq6$1...@news.dg.net.ua>, Eugene Mayevski

Look elsewhere for the causes of your access violations. What you have
described is a programming method that is used so often in Windows that if it
were as broken as you described, your newsreader wouldn't work long enough for
you to post your question.

When developing on Windows, it's tempting (and safe from ridicule) to blame
Windows for any and all crashes. It's important to remember, however, that
many people use Windows day in, day out, with no significant failures
whatever. If your application's crashing on you, and you're the developer of
that application, it's more than likely to be your fault.

Eugene Mayevski

unread,
Nov 5, 2002, 7:22:55 AM11/5/02
to
David Schwartz wrote:

> 2) Are you sure no other thread could possibly have been using that
> socket when you closed it?

Of course no. The socket is in blocking mode and is busy doing
something. I close the socket from other thread to cancel blocking call.

Sincerely yours,
Eugene Mayevski

Eugene Mayevski

unread,
Nov 5, 2002, 7:26:07 AM11/5/02
to
Alun Jones wrote:

>>Ok, how would one explain that Winsock crashes when the socket is closed
>>in another thread from where it was created? There is no "program

>>architecture" at all. Just one thread transferring the data and another
>>which closes the socket.


> Look elsewhere for the causes of your access violations. What you have
> described is a programming method that is used so often in Windows that if it
> were as broken as you described, your newsreader wouldn't work long enough for
> you to post your question.

This was a sample project which created a secondary thread. That thread
did extensive data transfer in blocking mode and the main thread closed
the socket. There is nothing there that can crash. Tested on WinXP home.

Sincerely yours,
Eugene Mayevski

Casper H.S. Dik

unread,
Nov 5, 2002, 10:41:31 AM11/5/02
to
al...@texis.com (Alun Jones) writes:

>There isn't a "thread-safe" implementation of sockets available on _any_
>platform if what you're talking about is two threads sending TCP data on one
>socket over the same brief time period.

Solaris sockets are thread-safe under that definition.

I.e., all the data gets out, it gets out in the order specified by
the oepration and no duplkicate data gets out nor do you get
OS crashes or application crashes. There's only a partial order of
data defined.

Whether these are useful semantics is unclear, but that's true for
many "thread safe" calls, including multiple writers to the same
fd.

Phil Frisbie, Jr.

unread,
Nov 5, 2002, 11:43:47 AM11/5/02
to

If you are so sure it is not your own coding error then why not either post the
code or take me up on my offer to look at at?

David Schwartz

unread,
Nov 5, 2002, 12:57:28 PM11/5/02
to
Eugene Mayevski wrote:

> This was a sample project which created a secondary thread. That thread
> did extensive data transfer in blocking mode and the main thread closed
> the socket. There is nothing there that can crash. Tested on WinXP home.

1) Were you using Winsock2?

2) When one thread closed the socket, are you absolutely positive that
no other thread could possibly have used that socket in any way shape or
form?

If the answer to either of these is "no", then that's your problem.

DS

David Schwartz

unread,
Nov 5, 2002, 1:28:22 PM11/5/02
to
Eugene Mayevski wrote:

> This was a sample project which created a secondary thread. That thread
> did extensive data transfer in blocking mode and the main thread closed
> the socket. There is nothing there that can crash. Tested on WinXP home.

Was the secondary thread still doing a data transfer when you closed
the socket from another thread? This can cause problems on *any*
platform.

Do not believe anyone who tells you that the blocking call is
guaranteeed to exit with some particular error indication. It is not.
Destroying a shared resource while it is in use is always an error.

DS

Casper H.S. Dik

unread,
Nov 5, 2002, 2:47:37 PM11/5/02
to
David Schwartz <dav...@webmaster.com> writes:

> Do not believe anyone who tells you that the blocking call is
>guaranteeed to exit with some particular error indication. It is not.
>Destroying a shared resource while it is in use is always an error.

The operating system should handle such a condition gracefully;
older versions of Solaris did not and you really don't want processes
hanging around in such cases waiting for fds that went away.

But for the apst couple of years, Solaris has handled this gracefully
and threads using file descriptors that are closed get notified,
usually using the EBADF error return for whatever they were doing.

Of course, this usually is a coding error, but some operating system
vendors believe that their OS should continue to run even if the
application progammer makes a mistake.

David Schwartz

unread,
Nov 5, 2002, 3:11:27 PM11/5/02
to
"Casper H.S. Dik" wrote:

> David Schwartz <dav...@webmaster.com> writes:

> > Do not believe anyone who tells you that the blocking call is
> >guaranteeed to exit with some particular error indication. It is not.
> >Destroying a shared resource while it is in use is always an error.

> The operating system should handle such a condition gracefully;

I don't believe that this is possible.

> older versions of Solaris did not and you really don't want processes
> hanging around in such cases waiting for fds that went away.

> But for the apst couple of years, Solaris has handled this gracefully
> and threads using file descriptors that are closed get notified,
> usually using the EBADF error return for whatever they were doing.

Really? Even if another thread gets the CPU, open another file
descriptor, and gets the same fd before the thread that was blocked gets
a chance to run?



> Of course, this usually is a coding error, but some operating system
> vendors believe that their OS should continue to run even if the
> application progammer makes a mistake.

I don't know of any OS where the OS itself actually crashes. However,
crashing the application is totally acceptible. IMO, this is a bug as
serious as one thread 'free'ing a chunk of memory while another thread
is using it.

DS

Casper H.S. Dik

unread,
Nov 5, 2002, 3:30:35 PM11/5/02
to
David Schwartz <dav...@webmaster.com> writes:

> Really? Even if another thread gets the CPU, open another file
>descriptor, and gets the same fd before the thread that was blocked gets
>a chance to run?

The close of that fd will not finish until the the fd is cleaned up.

Of course, threads that rememebr that the fd was really some other fd will
now have problems. But the application will continue onwards.

Eugene Mayevski

unread,
Nov 5, 2002, 3:40:23 PM11/5/02
to
Phil Frisbie, Jr. wrote:

> If you are so sure it is not your own coding error then why not either post the
> code or take me up on my offer to look at at?

1) It's Delphi code
2) It's a small part of the large product not yet released.

I am sure you can create such a test yourself -- connect a thread, a
socket in that thread and start sending data (lots of data). And another
thread will close the socket. After application is closed, an Access
Violation in the range of addresses that belong to Winsock, happens.

Sincerely yours,
Eugene Mayevski

Eugene Mayevski

unread,
Nov 5, 2002, 3:42:06 PM11/5/02
to
David Schwartz wrote:

And this is what I had :). There's an idea that a blocking call can be
cancelled from another thread by closing this socket. Now I know it's
not true :) (the idea was not mine, but written in FAQ for Delphi
library called Indy, which is now part of Delphi).

Sincerely yours,
Eugene Mayevski

Alun Jones

unread,
Nov 5, 2002, 3:49:50 PM11/5/02
to
In article <3dc82a6b$0$46605$e4fe...@news.xs4all.nl>, Casper H.S. Dik
<Caspe...@Sun.COM> wrote:
>David Schwartz <dav...@webmaster.com> writes:
>
>> Really? Even if another thread gets the CPU, open another file
>>descriptor, and gets the same fd before the thread that was blocked gets
>>a chance to run?
>
>The close of that fd will not finish until the the fd is cleaned up.

Non sequitur. David didn't say that the thread that was blocked was in a read
or write operation on the fd, therefore, the close will finish, the thread
will unblock, and will try to act on an fd that has been closed and then
reopened.

>Of course, threads that rememebr that the fd was really some other fd will
>now have problems. But the application will continue onwards.

.. and will now have one thread communicating merrily on the wrong socket /
file.

Because descriptors get re-used - and in most OS's, reused quickly - if more
than one thread gets to hear of a descriptor to the same object, and the
object isn't keeping a reference count of who has a descriptor to it, there's
always a possibility - a likelihood, even - that the descriptor will later
refer to an object that is not the right one.

Closing a socket in order to free up a potentially blocked socket operation
leads to code that leads to the above problem. The solution, if there is one,
is to ensure that no operation can possibly be in progress on a socket handle
at the time that you close it. You can do this by being single-threaded, or
you can do this by ensuring that your socket operations never block, or by
ensuring that every blocked operation is "timed out" at some point, and that
there is some signalling mechanism that prevents you from re-blocking on a
socket that's "about to close". The latter is significantately
complexificated.

Alun Jones

unread,
Nov 5, 2002, 3:49:48 PM11/5/02
to
In article <cHJx9.46662$Mb3.2...@bgtnsc04-news.ops.worldnet.att.net>,
"alex" <alex...@hotmail.com> wrote:
>If "They are guaranteed thread-safe in Winsock2", how about Winsock below 2?
>Will Winsock1 be thread-safe?
>
>In Winsock2, will send and receive in two thread be a problem?

Winsock 1 was specified before Windows had threads. Hence, it didn't consider
whether implementations should, or should not, be thread-safe.

As 32-bit threaded Windows came along, it was obvious that people would put
Winsock code into multi-threaded programs, but it was left up to the Winsock
implementation to choose whether to be thread-safe, or to put the onus back to
the application programmer. However, I'd be surprised if you were to find one
that didn't work for multi-threaded programs as you'd expect it to.

Alun Jones

unread,
Nov 5, 2002, 3:49:57 PM11/5/02
to
In article <aq9adc$1lu3$1...@news.dg.net.ua>, Eugene Mayevski
<maye...@eldos.org> wrote:
>Phil Frisbie, Jr. wrote:
>
>> If you are so sure it is not your own coding error then why not either post
> the
>> code or take me up on my offer to look at at?
>
>1) It's Delphi code

That's right, you're the only programmer here who understands Delphi. Oh,
sorry, I forgot the <sarcasm> tag.

>2) It's a small part of the large product not yet released.

As Phil says, either put up or go debug. If you can create a smallish sample
that demonstrates the problem, then the problem exists. Otherwise, it's with
your ancillary code.

>I am sure you can create such a test yourself -- connect a thread, a
>socket in that thread and start sending data (lots of data). And another
>thread will close the socket. After application is closed, an Access
>Violation in the range of addresses that belong to Winsock, happens.

As noted before, if this were the sum total of the flaw, then many other
applications than yours would be fragile beyond belief. Show us a sample and
we might believe you - or we might very well show you the error.

Either way, you win.

Right now, however, you're asking us to write a test for a behaviour that we
don't believe exists, when we've seen previous samples that would seem to
refute your suggestions. You won't persuade anyone to write code to
demonstrate something they believe is impossible!

Alun Jones

unread,
Nov 5, 2002, 3:49:54 PM11/5/02
to
In article <aq9agj$1lu3$2...@news.dg.net.ua>, Eugene Mayevski
<maye...@eldos.org> wrote:
>And this is what I had :). There's an idea that a blocking call can be
>cancelled from another thread by closing this socket. Now I know it's
>not true :) (the idea was not mine, but written in FAQ for Delphi
>library called Indy, which is now part of Delphi).

The idea of cancelling a blocking call by closing the socket is great for any
system where you can be sure that the socket's descriptor will not be
re-allocated before all references to the descriptor (in the application _and_
in the system) are dropped. Many client applications fit this profile (or can
be shoe-horned into it).

Blocking sockets may seem the obvious partner for multi-threaded code, but as
David has pointed out, there are some subtleties that can lead to migraines.

Alun Jones

unread,
Nov 5, 2002, 3:49:46 PM11/5/02
to
In article <3dc7e6ab$0$46602$e4fe...@news.xs4all.nl>, Casper H.S. Dik
<Caspe...@Sun.COM> wrote:
>al...@texis.com (Alun Jones) writes:
>
>>There isn't a "thread-safe" implementation of sockets available on _any_
>>platform if what you're talking about is two threads sending TCP data on one
>>socket over the same brief time period.
>
>Solaris sockets are thread-safe under that definition.
>
>I.e., all the data gets out, it gets out in the order specified by
>the oepration and no duplkicate data gets out nor do you get
>OS crashes or application crashes. There's only a partial order of
>data defined.

Let me get this straight:

If you have two threads, accessing the same TCP socket, and they both call
send(), the operating system will block the second thread that called send()
until it has queued up _all_ of the data from the first thread that called
send()?

>Whether these are useful semantics is unclear, but that's true for
>many "thread safe" calls, including multiple writers to the same
>fd.

It's thoroughly unclear that these are useful semantics, indeed, and one does
have to ask whether the question "are sockets thread-safe" wouldn't best be
answered with "if you need to know that, then your code is flawed".

Phil Frisbie, Jr.

unread,
Nov 5, 2002, 4:38:39 PM11/5/02
to
Eugene Mayevski wrote:
>
> Phil Frisbie, Jr. wrote:
>
> > If you are so sure it is not your own coding error then why not either post the
> > code or take me up on my offer to look at at?
>
> 1) It's Delphi code

Pascal was the first language I took in college, after I taught myself BASIC and
assembly in High School.

> 2) It's a small part of the large product not yet released.

A code sample should be fine.

> I am sure you can create such a test yourself -- connect a thread, a
> socket in that thread and start sending data (lots of data). And another
> thread will close the socket. After application is closed, an Access
> Violation in the range of addresses that belong to Winsock, happens.

Are you using Winsock API directly, or a wrapper?

> Sincerely yours,
> Eugene Mayevski

Phil Frisbie, Jr.

unread,
Nov 5, 2002, 4:51:11 PM11/5/02
to
David Schwartz wrote:
>
> Eugene Mayevski wrote:
>
> > This was a sample project which created a secondary thread. That thread
> > did extensive data transfer in blocking mode and the main thread closed
> > the socket. There is nothing there that can crash. Tested on WinXP home.
>
> Was the secondary thread still doing a data transfer when you closed
> the socket from another thread? This can cause problems on *any*
> platform.

Yes, that is part of the program architecture I tried to explain. That operation
is not normal, but closing the socket at other times on another thread is common
and trouble free.

> Do not believe anyone who tells you that the blocking call is
> guaranteeed to exit with some particular error indication. It is not.
> Destroying a shared resource while it is in use is always an error.

It depends on how you are using it (your program architecture).

For example, it is common to use multiple threads in a server application. You
might have one thread blocking for a few ms on select() waiting for received
data on multiple sockets, and another that sleeps for several seconds at a time
looking for stale connections to close. When that second thread closes a socket,
select() will unblock and return ENOTSOCK. There is no real error since you just
closed the socket and the handle is no longer valid.

Of course, there are many other ways to do this. You could simply flag the
socket to be closed and then close it later in your select() loop. Or you could
simply perform the stale connection check in that select() loop.

Casper H.S. Dik

unread,
Nov 6, 2002, 3:39:20 AM11/6/02
to
al...@texis.com (Alun Jones) writes:

>Let me get this straight:

>If you have two threads, accessing the same TCP socket, and they both call
>send(), the operating system will block the second thread that called send()
>until it has queued up _all_ of the data from the first thread that called
>send()?

That's not what I said; I said there's partial ordering defined.
(i.e., the data of the first wrte and second write are send in the proper
order, but no order is defined for the data of first and second write;
they can be interleaved.

For STREAMs, there's a guarantee that upto PIPE_MAX bytes are written
in one write.

>It's thoroughly unclear that these are useful semantics, indeed, and one does
>have to ask whether the question "are sockets thread-safe" wouldn't best be
>answered with "if you need to know that, then your code is flawed".

That might be one answer; but the idea of a monitor thread that closes
hanging connections has some appeal.

Fernando Gont

unread,
Nov 6, 2002, 5:56:20 AM11/6/02
to
On 06 Nov 2002 08:39:20 GMT, Casper H.S. Dik <Caspe...@Sun.COM>
wrote:

>>If you have two threads, accessing the same TCP socket, and they both call

>>send(), the operating system will block the second thread that called send()
>>until it has queued up _all_ of the data from the first thread that called
>>send()?
>That's not what I said; I said there's partial ordering defined.
>(i.e., the data of the first wrte and second write are send in the proper
>order, but no order is defined for the data of first and second write;
>they can be interleaved.

Sorry?

--
Fernando Gont
e-mail: fern...@ANTISPAM.gont.com.ar

[To send a personal reply, please remove the ANTISPAM tag]

Alun Jones

unread,
Nov 6, 2002, 7:23:54 PM11/6/02
to
In article <3dc8f1f...@News.CIS.DFN.DE>, arie...@softhome.net (Fernando
Gont) wrote:
>On 06 Nov 2002 08:39:20 GMT, Casper H.S. Dik <Caspe...@Sun.COM>
>wrote:
>
>>>If you have two threads, accessing the same TCP socket, and they both call
>>>send(), the operating system will block the second thread that called send()
>>>until it has queued up _all_ of the data from the first thread that called
>>>send()?
>>That's not what I said; I said there's partial ordering defined.
>>(i.e., the data of the first wrte and second write are send in the proper
>>order, but no order is defined for the data of first and second write;
>>they can be interleaved.
>
>Sorry?

I think he means that sockets is thread-safe unless you try to do something
that might cause them to be not so. :-)

Actually, he's illustrated exactly my point. It depends on what you mean by
"thread safe". If you're sending into one TCP socket on two threads, the data
may get interleaved - the socket stack is, therefore, not thread-safe. But
then, if you're going to do stupid stunts like that, then you're not looking
for any sane kind of thread-safety.

Notwithstanding Casper's obvious alliance to one stack, his definition of
"thread-safe" applies just as well to Windows as it does to his favoured
stack. It might not be the OP's definition of "thread-safe", though.

Alun Jones

unread,
Nov 6, 2002, 8:32:24 PM11/6/02
to
In article <3dc8d538$0$46608$e4fe...@news.xs4all.nl>, Casper H.S. Dik
<Caspe...@Sun.COM> wrote:
>al...@texis.com (Alun Jones) writes:
>
>>Let me get this straight:
>
>>If you have two threads, accessing the same TCP socket, and they both call
>>send(), the operating system will block the second thread that called send()
>>until it has queued up _all_ of the data from the first thread that called
>>send()?
>
>That's not what I said; I said there's partial ordering defined.

Actually, what you said was:

In article <3dc7e6ab$0$46602$e4fe...@news.xs4all.nl>, Casper H.S. Dik
<Caspe...@Sun.COM> wrote:
>al...@texis.com (Alun Jones) writes:
>
>>There isn't a "thread-safe" implementation of sockets available on _any_
>>platform if what you're talking about is two threads sending TCP data on one
>>socket over the same brief time period.
>
>Solaris sockets are thread-safe under that definition.

That sounds very much like what I posted. If you don't block, then data is
interleaved, and the sockets aren't "thread-safe".

>>It's thoroughly unclear that these are useful semantics, indeed, and one does
>>have to ask whether the question "are sockets thread-safe" wouldn't best be
>>answered with "if you need to know that, then your code is flawed".
>
>That might be one answer; but the idea of a monitor thread that closes
>hanging connections has some appeal.

There's plenty of things that have some appeal, but don't work.

Chris Pearson

unread,
Nov 7, 2002, 12:58:01 AM11/7/02
to
> Now I know it's not true :)

The closesocket doc says, "If this is the last reference to an underlying
socket, the associated naming information and queued data are discarded. Any
pending blocking, asynchronous calls issued by any thread in this process
are canceled without posting any notification messages."

The notions of reference counting, pending blocking calls and multiple
threads are all mentioned.

And how could you gracefully shutdown a (simplistic) server that used a
thread-per-blocking-socket architecture if you couldn't close the blocked
sockets from the SCM thread?

Run your program under a real debugger (one that can show the machine code,
registers, the call stack, and system symbols), wait for your crash, then
look at the stack -- your likely to find yourself staring at familiar code
:-)

-- CCP

"Eugene Mayevski" <maye...@eldos.org> wrote in message
news:aq9agj$1lu3$2...@news.dg.net.ua...

David Schwartz

unread,
Nov 7, 2002, 1:48:25 PM11/7/02
to
Chris Pearson wrote:

> The closesocket doc says, "If this is the last reference to an underlying
> socket, the associated naming information and queued data are discarded. Any
> pending blocking, asynchronous calls issued by any thread in this process
> are canceled without posting any notification messages."

It's not entirely clear what cancellation of a blocking socket call
entails.



> The notions of reference counting, pending blocking calls and multiple
> threads are all mentioned.

Yup.



> And how could you gracefully shutdown a (simplistic) server that used a
> thread-per-blocking-socket architecture if you couldn't close the blocked
> sockets from the SCM thread?

You can't. That you can is a myth. It's simply impossible. Consider a
thread doing this:

// some code
// a blocking socket call

There is no way another thread could possibly tell if the thread above
is in 'some code' or 'a blocking socket call'. There is no atomic 'set
flag and make blocking socket call' instruction.

So, if you close the socket in another thread just before the thread
above enters 'a blocking socket call', you cannot ensure that another
thread won't create a socket and get the same socket decriptor as the
thread above was about to perform a blocking socket call on. Now this
thread performs an operation on the *wrong* socket.

As I've said many times, it's an error to close a socket in one thread
while another thread is or might be using it. There is no way to do it
sanely.



> Run your program under a real debugger (one that can show the machine code,
> registers, the call stack, and system symbols), wait for your crash, then
> look at the stack -- your likely to find yourself staring at familiar code
> :-)

That's true.

DS

Vernon Schryver

unread,
Nov 7, 2002, 2:58:41 PM11/7/02
to
In article <3DCAB579...@webmaster.com>,
David Schwartz <dav...@webmaster.com> wrote:

>> The closesocket doc says, "If this is the last reference to an underlying
>> socket, the associated naming information and queued data are discarded. Any
>> pending blocking, asynchronous calls issued by any thread in this process
>> are canceled without posting any notification messages."
>
> It's not entirely clear what cancellation of a blocking socket call
>entails.

If you've ever looked at operating systems, it is entirely clear for
reasonable operating systems. Of necessity, the kernel or library code
that actually fiddles with a socket is single-threaded wherever the basic
common state of the socket is changed. You must protect the socket's
fundamental state from races. Whether the socket is "open" is its most
basic state. Part of that single-threading is almost certain to involve
every thread fiddling with a socket doing an equivalent to this dance:

socket_system_call()
{
if (validate_stuff() == failure)
return failure to caller;
loop: if (lock_socket() == failure
return failure to caller;
if (validate_more_stuff() == failure) {
unlock_socket();
return failure to caller;
}
case (try_to_do_stuff_like_connect_close_send_or_receive) {
failed: unlock_socket(); return failure to caller;
done: unlock_socket(); return success to caller;
partly_done:
unlock_socket();
block_on_something_or_other();
goto loop;
}
}

"Cancellation of a blocking socket call" is generally no more or less
than waking up any threads stuck in block_on_something(). If the
socket has disappeared out from under the thread, the normal validation
of the socket specifier (e.g. UNIX-style FD int or Winsock pointer)
will notice and return with some sort of error or other indication to
the application.

In other words, what do you suppose must happen when the entire
application is terminated by the operating system while some threads
are waiting for the socket? Somehow the system must shoot down all
of the threads, without leaving any orphan locks. The system's solution
to the problem of killing all threads in an application is likely to
imply a reasonable set of reasults for what must happen when one thread
closes a file descriptor or socket or otherwise trashes a resource in
use by another thread.


> ...


>> And how could you gracefully shutdown a (simplistic) server that used a
>> thread-per-blocking-socket architecture if you couldn't close the blocked
>> sockets from the SCM thread?
>
> You can't. That you can is a myth. It's simply impossible.

I think closing a socket out from under a blocked thread is nasty, but
"simply impossible" seems true only if you don't know how things work.

> Consider a
>thread doing this:
>
>// some code
>// a blocking socket call
>
> There is no way another thread could possibly tell if the thread above
>is in 'some code' or 'a blocking socket call'. There is no atomic 'set
>flag and make blocking socket call' instruction.

Actually, if you know how things work, you know that statement is usually
false. For example, if you can stop thread context switching
(e.g. pthread_setschedparam() in some POSIX threading), you might be
able to find other threads's stacks and see if they are in socket
system call wrappers. That's other than clean, but it is certainly
not "impossible," as reasonable debuggers demonstrate every time you
use them on threaded applications using blocking sockets.


> So, if you close the socket in another thread just before the thread
>above enters 'a blocking socket call', you cannot ensure that another
>thread won't create a socket and get the same socket decriptor as the
>thread above was about to perform a blocking socket call on. Now this
>thread performs an operation on the *wrong* socket.

That's true or false, depending on the the system. It's more likely
to be true than false, but there's no law of nature or programming
that mandates that socket specifiers (UNIX FDs or Winsock things) be
recycled. If FDs are not recycled, perhaps to minimize multi-CPU
locking in NUMA multi-processors, it's false. When it's true, it is
more likely to cause hard to find bugs elsewhere, such as when you
close resources hidden in libraries. Even when it is true, problems
due to such bugs are likely to be rare, because context switches are
unlikely to occur where they'll cause problems. A thread is more
likely to be blocked on a socket that is closed out from under it than
to be between uses of it.


> As I've said many times, it's an error to close a socket in one thread
>while another thread is or might be using it. There is no way to do it
>sanely.

> ...

If by "error" you mean "something any competent programmer spends a lot
of effort and significant CPU overhead to avoid," then you are right.
If you mean "impossible to make it work reliably," then you are wrong.


Vernon Schryver v...@rhyolite.com

Casper H.S. Dik

unread,
Nov 7, 2002, 4:02:12 PM11/7/02
to
v...@calcite.rhyolite.com (Vernon Schryver) writes:

>That's true or false, depending on the the system. It's more likely
>to be true than false, but there's no law of nature or programming
>that mandates that socket specifiers (UNIX FDs or Winsock things) be
>recycled. If FDs are not recycled, perhaps to minimize multi-CPU
>locking in NUMA multi-processors, it's false. When it's true, it is
>more likely to cause hard to find bugs elsewhere, such as when you
>close resources hidden in libraries. Even when it is true, problems
>due to such bugs are likely to be rare, because context switches are
>unlikely to occur where they'll cause problems. A thread is more
>likely to be blocked on a socket that is closed out from under it than
>to be between uses of it.

I'm pretty sure some part of some standard requires the kernel to
return the lowest fd available; so yes, they have to be reused and
yues, they are reused quickly.

(As you can get at the lowest fd in O(log(n)) time, this really isn't
much of a problem)

Vernon Schryver

unread,
Nov 7, 2002, 4:13:49 PM11/7/02
to
In article <3dcad4d3$0$46599$e4fe...@news.xs4all.nl>,

Casper H.S. Dik <Caspe...@Sun.COM> wrote:

> ...


>I'm pretty sure some part of some standard requires the kernel to
>return the lowest fd available; so yes, they have to be reused and
>yues, they are reused quickly.

I'm interested in which standard says that.
I've seen plenty of code that assumes that the numerically smallest
available FD will be used on the next open() The major complications
guaranteeing it or even defining the notion of "the lowest FD available"
in a non-trivial multi-processor makes me reluctant to write such code
without a handy chapter and verse.

I found a reference in http://www.standardml.org/Basis/posix-io.html
for dup() yielding the "lowest one available" but no definitions
of either "lowest" or "available."
There are similar vague words in
http://www.opengroup.org/onlinepubs/007904975/functions/open.html

Does close() make an FD "available" after it has been returned by open()?
How soon is must that availability happen in a multi-process with weak
consistency? Contemplate races for "available" in an NUMA multi-processor.
It should be possible or at least desirable for a process with no open
files have open() to race on two CPUs, with one open() ultimately
returning -1 and other returning 1.


Vernon Schryver v...@rhyolite.com

James Carlson

unread,
Nov 8, 2002, 8:07:59 AM11/8/02
to
v...@calcite.rhyolite.com (Vernon Schryver) writes:
> Does close() make an FD "available" after it has been returned by open()?

If you're into standards-based humor, consider the implications of
EINTR returned by close(2). If you get that error, is the fd still
open or has it been closed? The standards don't say. If it's still
open and you don't close it, you have an fd leak. If it's closed and
you try to reclose it, you can (and because of the 'lowest available'
logic, probably *will*) nuke the result of a simultaneous open by some
other thread.

You can't win. :-/

--
James Carlson, Solaris Networking <james.d...@east.sun.com>
SUN Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677

Eugene A. Zharkov

unread,
Nov 8, 2002, 10:05:04 AM11/8/02