Cannot assign requested address
Brian Hurt
Any ideas or suggestions?
Also, sorry for wasting the bandwidth.
Brian
John Cowan
A new client port number is assigned every time the client opens a TCP
connection, and there are only 65536 port numbers, so eventually you
will run out. I'm not familiar with how much control Java gives you
over the underlying C socket API, but you can share client port
numbers between sockets, as long as you make sure that two connections
*to the same server port on the same IP address* don't share port
numbers.
--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures
Brian Hurt
That's not the problem. We wrap around multiple times (exhaust all 30K or so port numbers and start over again) multiple times before we hit this problem. We're closing the sockets as fast as we're opening them, so there are always plenty of unused port numbers around (we actually never have more than 2-3 sockets open at any given point in time).
--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures
--
You received this message because you are subscribed to the Google Groups "JVM Languages" group.
To post to this group, send email to jvm-la...@googlegroups.com.
To unsubscribe from this group, send email to jvm-language...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jvm-languages?hl=en.
Robert Fischer
still consumed a port number, and there's no way to free the port
numbers for re-use?
How does SO_REUSEADDR/setReuseAddress(boolean) play into this?
~~ Robert.
On Wed, Feb 2, 2011 at 5:36 PM, John Cowan <johnw...@gmail.com> wrote:
Brian Hurt
How does SO_REUSEADDR/setReuseAddress(boolean) play into this?
Hmm. I wonder if this might be what I'm running into- I'm opening sockets at such speed, that all the available sockets still haven't timed out from their last use?
Brian
John Cowan
> Wait—let me get this straight. Even if you close a socket, you've
> still consumed a port number, and there's no way to free the port
> numbers for re-use?
No, not at all. If the socket has been closed long enough, the kernel
will free the port number, and it can be reallocated. I was simply
speculating that the OP was opening ports faster than he was closing
them, but apparently not.
> How does SO_REUSEADDR/setReuseAddress(boolean) play into this?
Normally the kernel waits a while after the socket is closed before
the port can be reused, so that any incoming packets from the remote
socket can be discarded. SO_REUSEADDR suppresses this delay, at a
risk of accepting a packet from a previous connection to the same
host.
Brian Hurt
> Wait—let me get this straight. Even if you close a socket, you'veNo, not at all. If the socket has been closed long enough, the kernel
> still consumed a port number, and there's no way to free the port
> numbers for re-use?
will free the port number, and it can be reallocated. I was simply
speculating that the OP was opening ports faster than he was closing
them, but apparently not.
I'm looking into this. It's possible, even probable, that my "rate of socket opening" does increase shortly before I hit the problem. It's not wrapping the port numbers that are the problem, it's wrapping them too fast.
This is especially relevant because it looks like the problem goes away after a second or two. So this is making sense. Thanks.
Brian
John Wilson
This does sound like a TIME_WAIT problem. The socket is supposed to be
held in this state (and is unavailable for reuse) for twice the
transit time of a packet across the network. In practice most
implementations use a fixed period of time (around 5 seconds). So if
you manage to open and close 30K sockets in the TIME_WAIT period then
you'll run out of sockets even if you have none open. Most TCP/IP
implementations have a way of tuning the TIME_WAIT period. Using
setReuseAddress(true) should work too.
John Wilson
John Wilson
Just some other general observations:
Your application seems to be pushing the TCP protocol kind of close to
its limits. You can mess with the TIME_WAIT setting and the problem
will go away but TIME_WAIT is there for a reason and it's possible
that you will encounter very rare, irreproducible failures (it may
also make you more vulnerable to malicious attack). You might consider
binding multiple IP adresses to your NIC and spreading the load over
them. This would give you a nice way of scaling the number of server
sockets available to you (30K * the number of IP adresses).
Opening and closing a server socket is quite expensive (both
computationally and in terms of network packets). If you are
transferring small amounts of data you might want to consider using
Datagrams. Datagrams have very low overhead but, of course, have
unreliable delivery semantics so you would have to be able to tolerate
(e.g. have idempotent protocols) or detect and correct delivery
failures.
Apologies if this is telling you things you know already:)
John Wilson