Building a Scalable (100K+ User) Socket Server in .Net
Chris Mullins
scalable socket server written in C#.
The SoapBox Server has recently been tested to well over 100k simultanous
users. This means it's handling 100k TCP connections on a Windows Platform
without any trouble at all.
I finally got around to writing a blog post that describes the different
architectures we've gone through, and the pros and cons of each. All of
these architectures scale fairly well, but there have been some surprising
results. These architectures have included using the system threadpool,
managing our own thread pool, doing work on the IOCP threads, and a few
other things thrown in for good luck.
I've seen so many people asking for guidance in this area, and I hope this
serves people as a good starting point.
http://makeashorterlink.com/?R2EE5246D
All feedback is welcome, either here or on the blog...
--
Chris Mullins
Coversant, Inc.
![Carl Daniel [VC++ MVP]'s profile photo](https://lh3.googleusercontent.com/a/default-user=s40-c)
Carl Daniel [VC++ MVP]
Thanks for sharing - great article!
-cd
Chris Mullins
>> [Writing a Scalable Socket Server]
>> http://makeashorterlink.com/?R2EE5246D
> Thanks for sharing - great article!
Any suggestions for how to improve things? The great thing about UseNet is
the wealth of knowledge out here, people wise. Getting hold of that
knowledge can be tough, but please, if you have suggestions fire away!
--
Chris Mullins
http://www.coversant.net/blogs/cmullins
Carl Daniel [VC++ MVP]
news:OgN1pJqp...@TK2MSFTNGP03.phx.gbl...
> "Carl Daniel [VC++ MVP]" wrote
>
>>> [Writing a Scalable Socket Server]
>>> http://makeashorterlink.com/?R2EE5246D
>
>> Thanks for sharing - great article!
>
> Any suggestions for how to improve things? The great thing about UseNet is
> the wealth of knowledge out here, people wise. Getting hold of that
> knowledge can be tough, but please, if you have suggestions fire away!
I think you're to the (really) hard part now - do as SQL Server does (as
discussed in a previous thread). You need to keep your threads from being
blocked and avoid context switches like the plague. Unfortunately, ADO.NET
doesn't help you out a lot there. It'd be interesting to see (as you've
already commented) how much more you'd get out at the high end just by using
lockfree queues on high-end systems while sticking with the locking queue on
low(er) end systems.
-cd
Chris Mullins
> It'd be interesting to see (as you've already commented) how much more
> you'd get out at the high end just by using lockfree queues on high-end
> systems while sticking with the locking queue on low(er) end systems.
My next topic will probably be just that. I wrote a mean little program that
is contention bound, and uses all the different types of locks. I ran this
on single x86, dualx86, single x64, dual x64, dual x64 with hyperthreading,
dual itanium2, quad itanium2, and 16 processor itanium2.
The resulting graphs are.... interesting.
Now I just need to find time to write that up...
![Jon Skeet [C# MVP]'s profile photo](http://lh3.googleusercontent.com/a-/ALV-UjUcZ7cLpd5PD7juf-mIIANWDJjBlFAMnXgv2fuTUtqywc6Q3AUmOvVJgg=s40-c)
Jon Skeet [C# MVP]
If you have the time to run the tests with my "feature-rich" locks, I'd
be really interested in seeing the results. See
http://www.pobox.com/~skeet/csharp/miscutil/usage/locking.html for
usage, download link etc.
It sounds like they wouldn't be appropriate for your situation, but it
would be good to get more diverse stats on the penalty for using my
code over the "plain" locking.
--
Jon Skeet - <sk...@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Chris Mullins
>
> If you have the time to run the tests with my "feature-rich" locks, I'd
> be really interested in seeing the results. See
> http://www.pobox.com/~skeet/csharp/miscutil/usage/locking.html for
> usage, download link etc.
>
I recently (just this week) took a long and deep look through Jeff Richters
"Power Threading" library. It's online at:
http://www.wintellect.com/Resources.aspx
(signing up for an account is free, then you can download the source).
As somebody who takes quite a bit of pride in my knowledge of multi-threaded
code, I have to say I was humbled.
I learned more about locking in 4 hours this past tuesday evening than I've
leared in a long, long time - possibly ever. Some of his source code
commends were... eye opening.
// NOTE: Just reading here (as compared to repeatedly calling Exchange)
// improves performance because writing forces all CPUs to update this value
// Don't do variable = value because reordered by compiler and/or CPU
There was stuff like this throughout - as well as stuff that I wish I could
wrap my head arond.
I quite liked his abstract ResourceLock implementation, and the concrete
variations on it.
Unfortunatly I can't use this library, as the licensing of it prevents it
from being run with Mono. I'll have to send him an email and see if I can
get a relaxed license version.
--
Chris Mullins
Carl Daniel [VC++ MVP]
Cool library - definitely some good stuff in there. Kinda short on
documentation though :) Makes for more interesting reading that way, I
guess.
-cd
Barry Kelly
> I recently (just this week) took a long and deep look through Jeff Richters
> "Power Threading" library. It's online at:
> http://www.wintellect.com/Resources.aspx
> (signing up for an account is free, then you can download the source).
>
> As somebody who takes quite a bit of pride in my knowledge of multi-threaded
> code, I have to say I was humbled.
Yes, memory models are mind-bending - especially when you bring compiler
optimizations into mind as well. Are we really certain that the JIT
compiler respects Thread.MemoryBarrier(), or does it just affect the
CPU? :) It's certainly enough to scare you away from playing with
sophisticated lock-free programming unless you've got a lot of time and
hardware to play with to prove and test correctness.
-- Barry