TCP server max number of sockets

[eran....@gmail.com]

Hi all,

I need to develop a TCP server which communicate with TCP clients.
There can be Tens of thousands client which the server should handle.
Is there any limit to the number of open connections in the server assuming my PC uses 8GBytes of RAM with Intel I7 CPU?

Does .Net limit the number of open connections?
What is the best server Architecture I should use?

Thanks for the help.

[bradbury9]

.NET does not limit the number of connections. MS Windows (and other operative systems) does. Check this link http://smallvoid.com/article/winnt-tcpip-max-limit.html

[eran....@gmail.com]

Hi bradbury9,

Is there any C# sample which show a TCP server should be written in order to support as much as possible connected clients at a time?

Thanks.

[bradbury9]

Personally I have never written a tcp server but the process should be pretty straightforward.

-Create a windows service that handles the incoming connections.
-When an incoming connection arrives dispatch it to another thread, maybe using async API (never used that API, but it is well documented and there are answers about taht topic in this newsgroup).

Maybe you should consider load balancing and creating a server farm

[Peter Duniho]

There's always a limit, but there should not be an explicit one.

For best performance, you should use one of the async patterns in .NET
(there are two different approaches for async in the Socket class, last I
checked), as they are implemented using I/O completion ports, which on
Windows is the correct way to have the best scalability. Other
implementations, such as using the Socket.Select() method or assigning an
individual thread per connection, will not be able to host nearly as many
connections efficiently as using the async patterns.

I'm not aware of any code examples that are specifically designed to
illustrate a scalable server, but MSDN does have code examples showing how
to use the async patterns.

The main different between the two async patterns is that the newer one
allows re-use of i/o-related objects, reducing load on the garbage
collector. Other than that, they should be roughly equivalent in theory (I
don't have first-hand experience with the newer pattern, but this is what
I've read).

Pete

[Arne Vajhøj]

On 2/26/2013 4:26 AM, eran....@gmail.com wrote:
> I need to develop a TCP server which communicate with TCP clients.
> There can be Tens of thousands client which the server should handle.
> Is there any limit to the number of open connections in the server assuming my PC uses 8GBytes of RAM with Intel I7 CPU?
>
> Does .Net limit the number of open connections?

TCP/IP limits the number.

The operating system or TCP/IP stack limits the number.

.NET may or may not limit all or certain operations based on how it
interacts with the native API's. I have never heard about such limits,
but that does not prove that such does not exist.

> What is the best server Architecture I should use?

Impossible to say with certainty based on so little information.

But as a rule of thumb:

1-500 clients : one thread per TCP connection [assuming a decent number
of CPU threads available]
501-5000 clients : fixed number of threads (much smaller than number
clients) handling all clients [.NET has rick API's for that]
5001- clients : switch to UDP

Arne

[Arne Vajhøj]

On 2/26/2013 9:10 AM, bradbury9 wrote:
> El martes, 26 de febrero de 2013 13:15:48 UTC+1, eran....@gmail.com

> escribi�:

>> On Tuesday, February 26, 2013 12:33:02 PM UTC+2, bradbury9 wrote:
>>> El martes, 26 de febrero de 2013 10:26:36 UTC+1,

>>> eran....@gmail.com escribi�:

>>>> I need to develop a TCP server which communicate with TCP
>>>> clients.
>>>>
>>>> There can be Tens of thousands client which the server should
>>>> handle.

>>>> What is the best server Architecture I should use?

>>> .NET does not limit the number of connections. MS Windows (and
>>> other operative systems) does. Check this link
>>> http://smallvoid.com/article/winnt-tcpip-max-limit.html

>> Is there any C# sample which show a TCP server should be written in
>> order to support as much as possible connected clients at a time?

> Personally I have never written a tcp server but the process should
> be pretty straightforward.
>
> -Create a windows service that handles the incoming connections.
> -When an incoming connection arrives dispatch it to another thread,
> maybe using async API (never used that API, but it is well documented
> and there are answers about taht topic in this newsgroup).

A thread per client will not work with so many clients.

Another paradigm is needed.

The various async API's are the .NET way of doing it.

> Maybe you should consider load balancing and creating a server farm

If the servers need to share state between clients, then it will
also complicate the application quite a bit.

Arne

[Arne Vajhøj]

On 2/26/2013 7:15 AM, eran....@gmail.com wrote:
> On Tuesday, February 26, 2013 12:33:02 PM UTC+2, bradbury9 wrote:
>> El martes, 26 de febrero de 2013 10:26:36 UTC+1, eran....@gmail.com

>> escribi�:

>>> Does .Net limit the number of open connections?
>>>
>>> What is the best server Architecture I should use?

>> .NET does not limit the number of connections. MS Windows (and
>> other operative systems) does. Check this link
>> http://smallvoid.com/article/winnt-tcpip-max-limit.html
>

> Is there any C# sample which show a TCP server should be written in
> order to support as much as possible connected clients at a time?

Maybe something like:

http://msdn.microsoft.com/en-us/library/fx6588te.aspx

Arne

[Peter Duniho]

On Tue, 26 Feb 2013 21:42:55 -0500, Arne Vajhøj wrote:

> [...]

> 5001- clients : switch to UDP

I'm skeptical that this is a good idea.

The only person I know personally first-hand who has written a large-scale
server architecture from scratch in .NET, he supports hundreds of thousands
of simultaneous clients with TCP using the async APIs (the first version).

UDP won't decrease network overhead. It probably will even increase it. It
for sure _will_ increase code complexity, along with design,
implementation, and maintenance costs. It _might_ reduce local resource
usage, depending on platform, but at the same time will likely increase the
required numbers of re-transmits for any given datagram, due to the lack of
per-client buffering (again, depending on platform...it's theoretically
possible for a network driver to maintain per-client buffering even for
UDP, but I'm not aware of any such implementation).

UDP is good for certain things. But I'm aware of no valid evidence to
suggest it's the right approach for increasing the number of clients a
server can support.

Frankly, the fact that HTTP is designed around TCP and that HTTP servers
are able to handle well above your "5000" number seems proof enough to me
that there's no need to change protocols just to support larger numbers of
clients. There _are_ web servers out there successfully dealing with far
larger numbers than that.

Yes, you need a machine configured to handle the number of connections you
want to support. But ensuring that is way easier to do than writing a
reliable network i/o component on top of UDP.

UDP is generally appropriate when your particular need doesn't require
reliable delivery of data; otherwise, TCP is the right solution. I will
grant that it's a common mistake to choose UDP because one thinks it's
faster, more efficient, etc. but in practice those things turn out to
generally not be true, nor worth the trade-off of an incorrect UDP-based
implementation that screws up your data.

Pete

[bradbury9]

El miércoles, 27 de febrero de 2013 03:49:44 UTC+1, Arne Vajhøj escribió:
> On 2/26/2013 9:10 AM, bradbury9 wrote:
>
> > El martes, 26 de febrero de 2013 13:15:48 UTC+1, eran....@gmail.com
>

> > escribió:

>
> >> On Tuesday, February 26, 2013 12:33:02 PM UTC+2, bradbury9 wrote:
>
> >>> El martes, 26 de febrero de 2013 10:26:36 UTC+1,
>

> >>> eran....@gmail.com escribió:

>
> >>>> I need to develop a TCP server which communicate with TCP
>
> >>>> clients.
>
> >>>>
>
> >>>> There can be Tens of thousands client which the server should
>
> >>>> handle.
>
>
>
> >>>> What is the best server Architecture I should use?
>
>
>
> >>> .NET does not limit the number of connections. MS Windows (and
>
> >>> other operative systems) does. Check this link
>
> >>> http://smallvoid.com/article/winnt-tcpip-max-limit.html
>
>
>
> >> Is there any C# sample which show a TCP server should be written in
>
> >> order to support as much as possible connected clients at a time?
>
>
>
> > Personally I have never written a tcp server but the process should
>
> > be pretty straightforward.
>
> >
>
> > -Create a windows service that handles the incoming connections.
>
> > -When an incoming connection arrives dispatch it to another thread,
>
> > maybe using async API (never used that API, but it is well documented
>
> > and there are answers about taht topic in this newsgroup).
>
>
>
> A thread per client will not work with so many clients.
>
>
>
> Another paradigm is needed.
>
>
>
> The various async API's are the .NET way of doing it.
>
>
>
> > Maybe you should consider load balancing and creating a server farm
>
>
>
> If the servers need to share state between clients, then it will
>
> also complicate the application quite a bit.
>
>
>
> Arne

Totally agree on the state issue, but the OP wants tons of connections AND use TCP so not an easy task without some sort of load balancing if tens of thousands connections are needed. Besides a server farm makes easy a contingency plan.

[Arne Vajhøj]

On 2/26/2013 10:57 PM, Peter Duniho wrote:
> On Tue, 26 Feb 2013 21:42:55 -0500, Arne Vajhøj wrote:
>
>> [...]
>> 5001- clients : switch to UDP
>
> I'm skeptical that this is a good idea.
>
> The only person I know personally first-hand who has written a large-scale
> server architecture from scratch in .NET, he supports hundreds of thousands
> of simultaneous clients with TCP using the async APIs (the first version).

There are plenty of examples using UDP out there. Documented examples.

> UDP won't decrease network overhead. It probably will even increase it.

It will decrease network overhead. But that was not the point.

> It
> for sure _will_ increase code complexity, along with design,
> implementation, and maintenance costs.

It uses the same event driven model as TCP without a thread per
connection, so no change for that part.

Some reliability features missing in UDP may need to be added, but
that will be rather trivial compared to all the other challenges
in a solution of that magnitude.

> It _might_ reduce local resource
> usage, depending on platform, but at the same time will likely increase the
> required numbers of re-transmits for any given datagram, due to the lack of
> per-client buffering (again, depending on platform...it's theoretically
> possible for a network driver to maintain per-client buffering even for
> UDP, but I'm not aware of any such implementation).
>
> UDP is good for certain things. But I'm aware of no valid evidence to
> suggest it's the right approach for increasing the number of clients a
> server can support.

????

Getting rid of the connection concept do avoid problems with max
connections rather effectively.

> Frankly, the fact that HTTP is designed around TCP and that HTTP servers
> are able to handle well above your "5000" number seems proof enough to me
> that there's no need to change protocols just to support larger numbers of
> clients. There _are_ web servers out there successfully dealing with far
> larger numbers than that.

I think you have misunderstood how HTTP works. HTTP connects, interact
and disconnect. Using keep-alive enables multiple interactions between
connect and disconnect.

Therefore N users accessing the web server does not mean
N concurrent TCP connections.

You can serve maybe 25000 users with just 2500 TCP connections.

Furthermore at least the most used web server (Apache) let connections
wait in the backlog queue until it is ready to process the request. I
suspect that IIS/ASP.NET does something similar.

HTTP and a socket client/server solution are simply very different
in how they do things, so you can not conclude much from HTTP.

> Yes, you need a machine configured to handle the number of connections you
> want to support. But ensuring that is way easier to do than writing a
> reliable network i/o component on top of UDP.
>
> UDP is generally appropriate when your particular need doesn't require
> reliable delivery of data; otherwise, TCP is the right solution. I will
> grant that it's a common mistake to choose UDP because one thinks it's
> faster, more efficient, etc. but in practice those things turn out to
> generally not be true, nor worth the trade-off of an incorrect UDP-based
> implementation that screws up your data.

What is needed for reliability depends on the problem domain.

And if there are high requirements, then there are well-tested
libraries available. I don't know if such exist for .NET though.

Arne

[Peter Duniho]

On Wed, 27 Feb 2013 20:47:07 -0500, Arne Vajhøj wrote:

> On 2/26/2013 10:57 PM, Peter Duniho wrote:
>> On Tue, 26 Feb 2013 21:42:55 -0500, Arne Vajhøj wrote:
>>
>>> [...]
>>> 5001- clients : switch to UDP
>>
>> I'm skeptical that this is a good idea.
>>
>> The only person I know personally first-hand who has written a large-scale
>> server architecture from scratch in .NET, he supports hundreds of thousands
>> of simultaneous clients with TCP using the async APIs (the first version).
>
> There are plenty of examples using UDP out there. Documented examples.

That's not the point. I never said one couldn't do it using UDP.

The point is that there are examples of TCP servers that work just fine
with just as many simultaneous connections.

>> UDP won't decrease network overhead. It probably will even increase it.
>
> It will decrease network overhead. But that was not the point.

There's no reason to believe UDP will decrease network overhead. Note that
by "overhead" I am referring to the cost of the network i/o relative to the
whole program (server). UDP itself may have less overhead for a given
amount of data (depending on usage patterns), but the same basic amount of
information (including data to identify the connection being handled) will
still need to move across the network, costing the server the same basic
overhead in network i/o.

>> for sure _will_ increase code complexity, along with design,
>> implementation, and maintenance costs.
>
> It uses the same event driven model as TCP without a thread per
> connection, so no change for that part.

???

TCP does not require a thread per connection. In fact, as has already been
made clear, the key to scalability on Windows is to use the I/O completion
port model, which is specifically about a small number of threads handling
a very large number of i/o operations.

> Some reliability features missing in UDP may need to be added, but
> that will be rather trivial compared to all the other challenges
> in a solution of that magnitude.

Writing a reliable implementation on top of UDP is not trivial at all. Lots
of programmers over the years have gotten, and will continue to get, it
wrong.

> [...]

>> UDP is good for certain things. But I'm aware of no valid evidence to
>> suggest it's the right approach for increasing the number of clients a
>> server can support.
>
> ????
>
> Getting rid of the connection concept do avoid problems with max
> connections rather effectively.

No. First, you are talking about solving a problem that does not exist. TCP
can handle way more than 5000 concurrent connections.

In any case, it simply shifts the cost from the network stack to the
program itself. If the program is going to depend on a connection-oriented
paradigm, that paradigm has to be implemented somewhere.

You can either let TCP handle it for you, or you can do it yourself. But it
has to live somewhere.

The bottom line is that existing TCP-based servers handle far greater
numbers of simultaneous connections than the 5000 you suggest for UDP.
Beyond that, there's simply no real-world indication that UDP is needed to
go beyond that 5000 threshold you suggest, nor is UDP an appropriate
solution when reliable delivery of data is needed.

Reliability in the network i/o is arguably as mission-critical as security,
and just as in the security world, it's foolish to try to reinvent the
wheel. Use the APIs that exist and are appropriate to your needs.

Show me a person who used UDP to get past 5000 concurrent connections in
their server, and I'll show you a person who just didn't bother to learn
how to implement the TCP version correctly.

Pete

(It's also worth pointing out that the real bottlenecks are likely not to
be in the network stack per se, but rather in the server's own logic in
handling clients, and in the network connection itself in supporting the
throughput requested. Handling more than 5000 clients is likely to be
highly dependent on the exact nature of the server's job, and the frequency
of the interactions with each client. But again, TCP itself is well up to
the job, regardless).

[Arne Vajhøj]

On 2/27/2013 9:40 PM, Peter Duniho wrote:
> On Wed, 27 Feb 2013 20:47:07 -0500, Arne Vajhøj wrote:
>
>> On 2/26/2013 10:57 PM, Peter Duniho wrote:
>>> On Tue, 26 Feb 2013 21:42:55 -0500, Arne Vajhøj wrote:
>>>
>>>> [...]
>>>> 5001- clients : switch to UDP
>>>
>>> I'm skeptical that this is a good idea.
>>>
>>> The only person I know personally first-hand who has written a large-scale
>>> server architecture from scratch in .NET, he supports hundreds of thousands
>>> of simultaneous clients with TCP using the async APIs (the first version).
>>
>> There are plenty of examples using UDP out there. Documented examples.
>
> That's not the point. I never said one couldn't do it using UDP.

You were skeptical that it was a good idea.

> The point is that there are examples of TCP servers that work just fine
> with just as many simultaneous connections.

Link ?

>>> UDP won't decrease network overhead. It probably will even increase it.
>>
>> It will decrease network overhead. But that was not the point.
>
> There's no reason to believe UDP will decrease network overhead. Note that
> by "overhead" I am referring to the cost of the network i/o relative to the
> whole program (server). UDP itself may have less overhead for a given
> amount of data (depending on usage patterns), but the same basic amount of
> information (including data to identify the connection being handled) will
> still need to move across the network, costing the server the same basic
> overhead in network i/o.

Not every network usage need all the features provides by TCP.

And if they don't need all the features they don't need all the
overhead.

And if they don't need the overhead they will be faster.

That is pretty simple logic.

>>> for sure _will_ increase code complexity, along with design,
>>> implementation, and maintenance costs.
>>
>> It uses the same event driven model as TCP without a thread per
>> connection, so no change for that part.
>
> ???
>
> TCP does not require a thread per connection. In fact, as has already been
> made clear,

Yes. Including by me.

But that is not what the above says.

It says that UDP use the same event driven model as TCP without a thread
per connection - unlike TCP with a thread per connection.

>> Some reliability features missing in UDP may need to be added, but
>> that will be rather trivial compared to all the other challenges
>> in a solution of that magnitude.
>
> Writing a reliable implementation on top of UDP is not trivial at all. Lots
> of programmers over the years have gotten, and will continue to get, it
> wrong.

That depends on the context/requirements.

In some cases it is very trivial. In some cases it is not.

>> [...]
>>> UDP is good for certain things. But I'm aware of no valid evidence to
>>> suggest it's the right approach for increasing the number of clients a
>>> server can support.
>>
>> ????
>>
>> Getting rid of the connection concept do avoid problems with max
>> connections rather effectively.
>
> No.

Yes.

> First, you are talking about solving a problem that does not exist. TCP
> can handle way more than 5000 concurrent connections.

It can. But it start to come with cost.

> In any case, it simply shifts the cost from the network stack to the
> program itself. If the program is going to depend on a connection-oriented
> paradigm, that paradigm has to be implemented somewhere.
>
> You can either let TCP handle it for you, or you can do it yourself. But it
> has to live somewhere.

A connection oriented paradigm requires something, but it does not
necessarily require the same overhead as TCP.

> The bottom line is that existing TCP-based servers handle far greater
> numbers of simultaneous connections than the 5000 you suggest for UDP.
> Beyond that, there's simply no real-world indication that UDP is needed to
> go beyond that 5000 threshold you suggest, nor is UDP an appropriate
> solution when reliable delivery of data is needed.

Not correct.

A non reliable protocol at transport layer does not mean unreliable
at the application level.

And here may be good reasons to put the reliability at the
application level instead of the transport level.

In fact there does not seem to be any correlation between
application level reliability and transport level reliability.

HTTP, FTP, SMTP, telnet does not seem more reliable than
DNS, NTP and NFS to me.

Arne

[Peter Duniho]

On Wed, 27 Feb 2013 22:20:01 -0500, Arne Vajhøj wrote:

>>> There are plenty of examples using UDP out there. Documented examples.
>>
>> That's not the point. I never said one couldn't do it using UDP.
>
> You were skeptical that it was a good idea.

Yes. Why reinvent the wheel for no good reason? That it can be done is no
good reason, in and of itself.

>> The point is that there are examples of TCP servers that work just fine
>> with just as many simultaneous connections.
>
> Link ?

I'm not going to spend a lot of time digging up references you could
yourself. For that matter, you could easily just test it yourself.

However, it's not too hard to find references to the work done by Chris
Mullins (who used to post here frequently). His messaging server SoapBox is
a good example of a highly scalable TCP implementation built using .NET.
Here are some references (including an archived copy of a thread from this
newsgroup):
http://www.coversant.net/products/soapbox-server
http://web.archive.org/web/20080209233229/http://www.coversant.net/Coversant/Blogs/tabid/88/EntryID/10/Default.aspx
http://bytes.com/topic/net/answers/511005-building-scalable-100k-user-socket-server-net

He used to have the article on his company's web site, but the links I
found for it are no longer valid. Surely you can find them if you feel they
are important.

> Not every network usage need all the features provides by TCP.

Completely different issue. Don't change the subject.

We are talking about a scenario where the reliability of TCP is needed. If
it weren't, one wouldn't/shouldn't be using TCP in the first place.

> [...]

>>> It uses the same event driven model as TCP without a thread per
>>> connection, so no change for that part.
>>
>> ???
>>
>> TCP does not require a thread per connection. In fact, as has already been
>> made clear,
>
> Yes. Including by me.
>
> But that is not what the above says.
>
> It says that UDP use the same event driven model as TCP without a thread
> per connection - unlike TCP with a thread per connection.

???

First you say that you didn't write that TCP requires a thread per
connection. Then you compare UDP to using "TCP with a thread per
connection".

You are making no sense.

The only reason to contrast UDP without a "thread per connection" to TCP
_with_ a "thread per connection" is if you actually need a thread per
connection in TCP. Otherwise, it's like saying "UDP is way better on my
128GB machine than TCP is on my 128MB machine".

There's no need to switch to UDP just to get away from having one thread
per connection. So why even mention it? It has nothing to do with whether
UDP is or is not better than TCP for handling large numbers of connections
(or clients, in the UDP case).

Pete

[Arne Vajhøj]

On 2/27/2013 11:34 PM, Peter Duniho wrote:
> On Wed, 27 Feb 2013 22:20:01 -0500, Arne Vajhøj wrote:
>
>>>> There are plenty of examples using UDP out there. Documented examples.
>>>
>>> That's not the point. I never said one couldn't do it using UDP.
>>
>> You were skeptical that it was a good idea.
>
> Yes. Why reinvent the wheel for no good reason? That it can be done is no
> good reason, in and of itself.

Well - it was not without reason. It was to avoid having thousands or
tens of thousands of TCP contexts.

>> Not every network usage need all the features provides by TCP.
>
> Completely different issue. Don't change the subject.

????

It is the core of the debate.

> We are talking about a scenario where the reliability of TCP is needed.

No. We are talking about OP's question. He did not require the
reliability of TCP in his question.

> If
> it weren't, one wouldn't/shouldn't be using TCP in the first place.

Which is exactly my point.

But unlike you I am not convinced that everybody asking a question
about TCP have carefully considered the UDP option.

>>>> It uses the same event driven model as TCP without a thread per
>>>> connection, so no change for that part.
>>>
>>> ???
>>>
>>> TCP does not require a thread per connection. In fact, as has already been
>>> made clear,
>>
>> Yes. Including by me.
>>
>> But that is not what the above says.
>>
>> It says that UDP use the same event driven model as TCP without a thread
>> per connection - unlike TCP with a thread per connection.
>
> ???
>
> First you say that you didn't write that TCP requires a thread per
> connection. Then you compare UDP to using "TCP with a thread per
> connection".
>
> You are making no sense.

I think you are just slow to get the point.

TCP with a thread per client is a convenient programming model.

TCP or UDP without a thread per client is slightly more work,
but it is the same model for TCP and UDP.

> There's no need to switch to UDP just to get away from having one thread
> per connection. So why even mention it? It has nothing to do with whether
> UDP is or is not better than TCP for handling large numbers of connections
> (or clients, in the UDP case).

See above.

Arne