send(), sendmsg(), sendto() not thread-safe
Mark A Smith
I discovered that in some cases, send(), sendmsg(), and sendto() are not
thread-safe. Although the man page for these functions does not specify
whether these functions are supposed to be thread-safe, my reading of the
POSIX/SUSv3 specification tells me that they should be. I traced the
problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
up a small page to describe in more detail my findings. You can find it at:
http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .
Thanks,
Mark A. Smith
PS. I am using the term "thread" in the general sense, this is a problem
independent of pthreads, etc. The problem occurs when two processes
(whether or not they share an address space) send on the same socket (and
some other low-resource conditions exist).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
H. Peter Anvin
By author: Mark A Smith <mark...@us.ibm.com>
In newsgroup: linux.dev.kernel
>
> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .
>
> Thanks,
> Mark A. Smith
>
> PS. I am using the term "thread" in the general sense, this is a problem
> independent of pthreads, etc. The problem occurs when two processes
> (whether or not they share an address space) send on the same socket (and
> some other low-resource conditions exist).
>
User error. Writes onto a streaming socket (or a pipe) are
thread-safe, *but not necessarily atomic*, if the size exceeds PIPE_BUF.
If you want atomicity you either have to do your own locking, or use a
DGRAM or SEQPACKET socket.
-hpa
David S. Miller
Date: Mon, 15 May 2006 14:39:06 -0700
> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .
I don't understand why the desire is so high to ensure that
individual threads get "atomic" writes, you can't even ensure
that in the general case.
Only sloppy programs that don't do their own internal locking hit into
issues in this area.
From your findings, the vast majority of systems you investigated do
not provide "atomic" thread safe write semantics over TCP sockets.
And frankly, BSD defines BSD socket semantics here not some wording in
the POSIX standards.
Finally, this discussion belongs on the networking development mailing
list, net...@vger.kernel.org, not linux-kernel.
David Schwartz
> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can
> find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .
>
> Thanks,
> Mark A. Smith
You are confusing thread-safety with atomicity.
DS
Christoph Hellwig
>
> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .
Please don't confuse thread safety with atomicy, thanks.