Blocking Socket question in Server Implementation
k3xji
I am trying to implement a Game Server which will serve 0-5000 clients
at a time. I am thinking to implement it in Python for to develop it
faster.
After days of reviewing different design approaches, I came up with
the following:
- Sockets are blocking, and we WILL have only one SOCKET connection to
client.
- Each client's request is processed in a separate thread.
- When a client requests something, the request is put in the JobQueue
of another thread. And this thread is responsible to process it.(This
is a MUST, I do not want to rely on the client thread itself to
process the Job)Here is my question arise.
As we have only one connection to the client, it is possible to have
THREAD A called socket.recv() and at the same time THREAD B called
socket.send().
Is it safe use send()/recv() simultaneously from different threads for
the same socket. And please note that addiion to THREAD A and B there
is a high chance that THREAD C might call socket.send() which makes
multiple call to recv()/send()/send() on the same socket.
Is multiple thread/one socket is not a reliable method for these
situation, which IS? And, why?
Different design suggestions?
Thanks all.
Peter Duniho
> [...]
> As we have only one connection to the client, it is possible to have
> THREAD A called socket.recv() and at the same time THREAD B called
> socket.send().
>
> Is it safe use send()/recv() simultaneously from different threads for
> the same socket.
Yes, that's fine.
> And please note that addiion to THREAD A and B there
> is a high chance that THREAD C might call socket.send() which makes
> multiple call to recv()/send()/send() on the same socket.
No, that's not fine. You need to figure out a way so that you handle
synchronization of your sends to ensure the data goes out the way you want.
As for your other comments, at 5000 clients, you're going to find "one
thread per client" too slow, unless your "game server" is for something
simple that doesn't involve having to actually communicate with any given
client very often. Ideally, you'd use IOCP.
Serializing your client requests to a single thread might or might not be
appropriate, from a performance and reliability point of view. Suffice to
say, that decision _might_ be hamstringing you the same way that your "one
thread per client" decision surely has.
Pete
k3xji
question:)
> > And please note that addiion to THREAD A and B there
> > is a high chance that THREAD C might call socket.send() which makes
> > multiple call to recv()/send()/send() on the same socket.
>
> No, that's not fine. You need to figure out a way so that you handle
> synchronization of your sends to ensure the data goes out the way you want.
That may not be a point. Let me explain this way, I am not caring
about THREAD B and C 's send synch. Thread B is actually sending the
msg/game data which is critical to the game server. However, THREAD C
is only an AUDIT thread which is constantly traversing the clients to
check if they are alive and if they are valid for some sort of
security. So from design perspective it is not important for me if
THREAD C's send operation is succeed before/after than the THREAD B. I
just want to know if this kind of implementation may cause race
conditions or some sort of problems in the OS level.
>Serializing your client requests to a single thread might or might not be
>appropriate, from a performance and reliability point of view. Suffice to
>say, that decision _might_ be hamstringing you the same way that your "one
>thread per client" decision surely has.
Can you give a simple example on how to implement X Clients with Y
Threads where Y is greater than 1 and smaller than X? I mean, if we
have 8 clients how to use let's say 2 threads to handle them, what
will I gain/lose from such design compared to the one thread/one
client alternative?
Regards
Peter Duniho
> [...] So from design perspective it is not important for me if
> THREAD C's send operation is succeed before/after than the THREAD B. I
> just want to know if this kind of implementation may cause race
> conditions or some sort of problems in the OS level.
It's not an issue of the send operation happening before or after. It's
that the data can get mixed up.
> [...]
> Can you give a simple example on how to implement X Clients with Y
> Threads where Y is greater than 1 and smaller than X? I mean, if we
> have 8 clients how to use let's say 2 threads to handle them, what
> will I gain/lose from such design compared to the one thread/one
> client alternative?
It depends on how you implement it. However, IOCP is designed to allow
just this scenario, with the most efficient use of CPU time. In
particular, using IOCP the operating system will do its best to ensure
that a thread that is already running will continue to run rather than
being preempted by some other thread.
This newsgroup has had a lot of discussion of IOCP over the years, and of
course the MSDN web site has lots of information on the topic as well.
Google can help you find the relevant mentions for more specifics.
Pete
David Schwartz
> Hi all,
>
> I am trying to implement a Game Server which will serve 0-5000 clients
> at a time. I am thinking to implement it in Python for to develop it
> faster.
It will be very hard to service 5,000 clients in python. Maybe modern
hardware is so fast that it can manage.
> After days of reviewing different design approaches, I came up with
> the following:
>
> - Sockets are blocking, and we WILL have only one SOCKET connection to
> client.
> - Each client's request is processed in a separate thread.
That's a bad idea. That means in order to, say, receive a single
character from each of 2,500 clients, you will have to switch thread
contexts around 2,500 times. That's stupid and massively inefficient.
> - When a client requests something, the request is put in the JobQueue
> of another thread. And this thread is responsible to process it.(This
> is a MUST, I do not want to rely on the client thread itself to
> process the Job)Here is my question arise.
That's a bad idea too. All the threads are the same. There is no
rational reason to care which thread does the job. If it makes you
feel better, that thread is not a "client thread" while it's
processing the job.
Don't think of a thread as having an identity for its entire life. A
"client thread" is one that is currently running the client code. If
it later runs some other code, it's just not a "client thread"
anymore. This way, you avoid the situation where the "wrong thread" is
running and you have to hand stuff off and do more context switches.
Make whatever thread is already running the right thread - by design.
> As we have only one connection to the client, it is possible to have
> THREAD A called socket.recv() and at the same time THREAD B called
> socket.send().
That's fine.
> Is it safe use send()/recv() simultaneously from different threads for
> the same socket.
Yes.
> And please note that addiion to THREAD A and B there
> is a high chance that THREAD C might call socket.send() which makes
> multiple call to recv()/send()/send() on the same socket.
That's illegal for TCP and can cause unpredictable results.
> Is multiple thread/one socket is not a reliable method for these
> situation, which IS? And, why?
> Different design suggestions?
Use 'poll' to see which sockets need to be read from. That way you
won't need 5,000 threads to serve 5,000 clients. Try to rig things so
that at thread can keep running and doing more and more work until
there's no work left to be done. Use multiple threads to keep multiple
processors busy to increase concurrency. Don't keep context on the
stack across network operations.
DS
k3xji
> It will be very hard to service 5,000 clients in python. Maybe modern
> hardware is so fast that it can manage.
I am doubtful if the implementation of this in C will do much better.
As far as I know, Context Switch is an expensible opration enough to
make no difference between a native language or dynamic language. I
mean the bottleneck for the server will be the Context Switch itself
in the OS Level, not the execution model(interpreter or compiler).
> > - Sockets are blocking, and we WILL have only one SOCKET connection to
> > client.
> > - Each client's request is processed in a separate thread.
>
> That's a bad idea. That means in order to, say, receive a single
> character from each of 2,500 clients, you will have to switch thread
> contexts around 2,500 times. That's stupid and massively inefficient.
Ok. You have a point. But I have my points, too:)
> > - When a client requests something, the request is put in the JobQueue
> > of another thread. And this thread is responsible to process it.(This
> > is a MUST, I do not want to rely on the client thread itself to
> > process the Job)Here is my question arise.
>
> That's a bad idea too. All the threads are the same. There is no
> rational reason to care which thread does the job. If it makes you
> feel better, that thread is not a "client thread" while it's
> processing the job.
>
> Don't think of a thread as having an identity for its entire life. A
> "client thread" is one that is currently running the client code. If
> it later runs some other code, it's just not a "client thread"
> anymore. This way, you avoid the situation where the "wrong thread" is
> running and you have to hand stuff off and do more context switches.
> Make whatever thread is already running the right thread - by design.
No, I don't accept that. I use JobServer because I don't want "client
thread"
to block on the send() calls. Maybe I am using bad names for this
purpose.
Let "Client thread" be the RecvThread for that specific client and the
JobServer be the SendThread
for all clients. Think of the following scenario:
- Client A sends message to all users in the lobby.
- Client thread understands the command and sends the message to all
users. While this is happening
client is blocked for any incoming requests, I donot want to have
that. Suppose this client is playing on
Game Table 1, this means one message send operation to all will
block the gameplay in that table.
Each client operation should not block itself to recv incoming
requests.
If there is better way to accomplish this, I am eager to hear
that...
> > Is multiple thread/one socket is not a reliable method for these
> > situation, which IS? And, why?
> > Different design suggestions?
>
> Use 'poll' to see which sockets need to be read from. That way you
> won't need 5,000 threads to serve 5,000 clients. Try to rig things so
> that at thread can keep running and doing more and more work until
> there's no work left to be done. Use multiple threads to keep multiple
> processors busy to increase concurrency. Don't keep context on the
> stack across network operations.
That's a good idea. But I am doubtful,on how to implement the 'poll'
mechanism.
You mean IOCP or constantly traverse all the clients to see if they
have any request?
If traverse all the clients for a request and if the socket is
blocking then you mean wait for
a request from a client until a specific timeout occurs. Like:
while(cli = clients.next):
if cli.socket.recv(1) == TIMEOUT:
Continue;
else
ProcessCommand()
Thanks.
David Schwartz
> On Aug 29, 1:58 am, David Schwartz <dav...@webmaster.com> wrote:
> > It will be very hard to service 5,000 clients in python. Maybe modern
> > hardware is so fast that it can manage.
> I am doubtful if the implementation of this in C will do much better.
16,000 is a breeze in C. We've been doing 16,000 in C++ on Windows and
UNIX for almost a decade now.
> As far as I know, Context Switch is an expensible opration enough to
> make no difference between a native language or dynamic language. I
> mean the bottleneck for the server will be the Context Switch itself
> in the OS Level, not the execution model(interpreter or compiler).
Right, that's why you should design server applications to minimize
the number of context switches. The situation you want to minimize is
this:
1) A thread is running.
2) There's work to be done.
3) The thread that's running cannot do any useful work.
Because this forces a context switch.
> No, I don't accept that. I use JobServer because I don't want "client
> thread"
> to block on the send() calls.
It can't, because while it's making those send calls, it's not a
client thread!
> Maybe I am using bad names for this
> purpose.
> Let "Client thread" be the RecvThread for that specific client and the
> JobServer be the SendThread
> for all clients. Think of the following scenario:
Okay.
> - Client A sends message to all users in the lobby.
> - Client thread understands the command and sends the message to all
> users. While this is happening
> client is blocked for any incoming requests,
Why is client blocked for any incoming requests? Because it now
doesn't have a client thread? That's fine -- just take one of the
threads that's not doing anything (or create a new one) to be its
client thread. Let the thread that's already running do the work that
you already need to be done.
Yes, there might be more work in the future. But that's not the
problem now.
> I donot want to have
> that. Suppose this client is playing on
> Game Table 1, this means one message send operation to all will
> block the gameplay in that table.
I don't see why.
> Each client operation should not block itself to recv incoming
> requests.
Right, so while some thread is doing a client operation, other threads
can receive incoming requests. That's why you have threads in the
first place! So one thread can do X, and another thread can do Y. But
make it the most convenient thread, not the most inconvenient.
Thread 1 receives a command from client A. This thread is running and
has the command. It's the absolute best possible thread to process
that command -- and some thread has to do it. So let this thread
process the command. Now, it's not a client thread anymore. Okay, you
are short one client thread. So what? You can keep a pool of spare
threads and make one of them the new client thread.
> If there is better way to accomplish this, I am eager to hear
> that...
A thread is a "client thread" while it's running the client code, not
for its entire life. Let a thread do whatever work needs to be done
when it's running. And if you have something that needs to be done
that no thread is doing, let the first thread that is not busy do it.
> > Use 'poll' to see which sockets need to be read from. That way you
> > won't need 5,000 threads to serve 5,000 clients. Try to rig things so
> > that at thread can keep running and doing more and more work until
> > there's no work left to be done. Use multiple threads to keep multiple
> > processors busy to increase concurrency. Don't keep context on the
> > stack across network operations.
> That's a good idea. But I am doubtful,on how to implement the 'poll'
> mechanism.
> You mean IOCP or constantly traverse all the clients to see if they
> have any request?
> If traverse all the clients for a request and if the socket is
> blocking then you mean wait for
> a request from a client until a specific timeout occurs. Like:
>
> while(cli = clients.next):
> if cli.socket.recv(1) == TIMEOUT:
> Continue;
> else
> ProcessCommand()
>
> Thanks.
Keep a table of clients and a queue of received and outbound messages
for each. Also keep a 'last heard time'. Periodically, scan the 'last
heard time' to timeout inactive clients.
Call 'poll'. Check for readability on all sockets and writability on
those with non-empty outbound queues. When you discover sockets
readable or writable, do the I/O to and from the queues, wake your
army of threads to process the new received messages (if any), and
call 'poll' again.
DS