Max number of threads that can be used?
Sonny
http://groups.google.com.ph/group/comp.programming.threads/browse_thread/thread/a733e05b0dc924b1?tvc=2&hl=tl
. I'm still doing the project, but this time i'm using a thread per
client model. For the recap, i'm supposed to create a middleware
program that accepts a message, parse this message then place it in a
database. It will then wait(loop) for the database update, format the
data then send it back. Actually the program functionality is
complete. It is working fine except like what I posted before it
should be capable of handling about 1600 concurrent connections. It
was tested on a pc with 256MB RAM and no major problems were
encountered if there were about 300 clients. However beyond 300 I'm
experiencing trouble creating (p)threads, (cannot allocate memory). I
have used valgrind, efence and gdb for debugging, and somehow cleaned
the program for memory leaks. Basically the suspect is the hardware
limit.
Now the server is running in Dual Core with 8GB of memory. Now it can
handle only about 350 connections. I always have database connection
problems beyond that, usually lost connection to MySQL server during
query, then MYSQL server has gone away. Running GDB with the core
dump, i usually encounter these 2:
Program terminated with signal 6, Aborted.
Cannot access memory at address 0x3052515988
or
Program terminated with signal 11, Segmentation fault.
Cannot access memory at address 0x3052515988
I don't know how to debug this kind of error. What would you suggest/
recommend?
Ian Collins
> I previously posted about threaded server using select(). See
> http://groups.google.com.ph/group/comp.programming.threads/browse_thread/thread/a733e05b0dc924b1?tvc=2&hl=tl
> client model. For the recap, i'm supposed to create a middleware
> program that accepts a message, parse this message then place it in a
> database. It will then wait(loop) for the database update, format the
> data then send it back. Actually the program functionality is
> complete. It is working fine except like what I posted before it
> should be capable of handling about 1600 concurrent connections. It
> was tested on a pc with 256MB RAM and no major problems were
> encountered if there were about 300 clients. However beyond 300 I'm
> experiencing trouble creating (p)threads, (cannot allocate memory). I
> have used valgrind, efence and gdb for debugging, and somehow cleaned
> the program for memory leaks. Basically the suspect is the hardware
> limit.
>
will grind to a halt spending all its time in context switches. They
way you are going, you'll have to use a 16 core machine with 64GB of RAM
just to crawl along.
--
Ian Collins.
Torsten Robitzki
> I don't know how to debug this kind of error. What would you suggest/
> recommend?
It may be that you are running out of virtual address space. If so:
- changing the stack size of the thread will decrease the used address space
- switch to 64 bit OS
- Change the architecture of your program
How do you handle data base connections? Does every thread create a own
connection? If so:
- Use a connection pool with a reasonable, limited amount of connections
Maybe you got a helpful error code from a called function, but you
ignored it?
best regards
Torsten
Markus Elfring
> handle only about 350 connections.
Did you try any advices from the previous discussion "threaded server using
select()" or the well-known document "The C10K problem" for your software?
http://kegel.com/c10k.html
Regards,
Markus
Sonny
> - changing the stack size of the thread will decrease the used address space
> - switch to 64 bit OS
> - Change the architecture of your program
I'm considering using threaded pool, but I really don't how to
implement it. The program's functionality is already complete using a
one thread per client. The clients doesn't need to communicate with
each other. I need to have a copy of the socket descriptor for each
thread since it will return a response. I think I'm having a problem
in the waiting part( checking for an update in the database), that is
where the clogging is happening. I'm encountering "commands out of
sync" here
Here is the part of my waiting loop
for (;;)
{
pthread_mutex_lock(&mutex2); // lock
freeIdx2=GetFreeConnection(); // get free database
connector
selectdb(&myconn[freeIdx2], queryString,
strlen(queryString), &res_set);
pthread_mutex_unlock(&mutex2);
if (!res_set)
{
perror("Query error");
}
else if ( (numrow=mysql_num_rows(res_set)) == 0)
{
if ( strcmp(cpdata.actionID,"04") == 0) // just
checking for a flag
{
if (function1(sendBuffer, insert_id));
}
else
{
if (function2(sendBuffer, insert_id));
}
break;
}
else
sleep(1); // sleep 1 second
mysql_free_result(res_set); // this is where GDB points to
when i'm encountering commands out of sync
}
Each client/thread will wait until the entry is deleted(processed).
> How do you handle data base connections? Does every thread create a own
> connection? If so:
> - Use a connection pool with a reasonable, limited amount of connections
I opened 10 database connections. I have a function that looks for an
unused connection. Do you think the problem is with MySQL, can't it
have concurrent connections?
> Did you try any advices from the previous discussion "threaded server using
> select()" or the well-known document "The C10K problem" for your software?
> http://kegel.com/c10k.html
I'm trying to suggest to use 6 PC's =). I like the idea of threaded
pool, but unfortunately i'm just a beginner in multihreading. Do you
have other references besides W. Richard Steven's books?
Torsten Robitzki
>>- Change the architecture of your program
>
> I'm considering using threaded pool, but I really don't how to
> implement it. The program's functionality is already complete using a
> one thread per client. The clients doesn't need to communicate with
> each other. I need to have a copy of the socket descriptor for each
> thread since it will return a response. I think I'm having a problem
> in the waiting part( checking for an update in the database), that is
> where the clogging is happening. I'm encountering "commands out of
> sync" here
> Here is the part of my waiting loop
>
> for (;;)
> {
> pthread_mutex_lock(&mutex2); // lock
> freeIdx2=GetFreeConnection(); // get free database
> connector
If you have a limited number of database connection, I would expect the
call to GetFreeConnection() to block until there is a database
connection available. Holding a mutex around a blocking function is
seldom a good idea and btw.: bad mutex name.
> selectdb(&myconn[freeIdx2], queryString,
> strlen(queryString), &res_set);
> pthread_mutex_unlock(&mutex2);
>
> if (!res_set)
> {
> perror("Query error");
> }
> else if ( (numrow=mysql_num_rows(res_set)) == 0)
> {
> if ( strcmp(cpdata.actionID,"04") == 0) // just
> checking for a flag
> {
> if (function1(sendBuffer, insert_id));
> }
> else
> {
> if (function2(sendBuffer, insert_id));
> }
> break;
> }
> else
> sleep(1); // sleep 1 second
Isn't there something more appropriate to wait for? Incoming requests
for example?
> mysql_free_result(res_set); // this is where GDB points to
> when i'm encountering commands out of sync
> }
I really can't see what you are doing there. You should use better names
for your data and functions.
>>How do you handle data base connections? Does every thread create a own
>>connection? If so:
>>- Use a connection pool with a reasonable, limited amount of connections
>
> I opened 10 database connections. I have a function that looks for an
> unused connection. Do you think the problem is with MySQL, can't it
> have concurrent connections?
Maybe, I've never used mysql in such a setting. But mysql comes with a
quit good documentation ;-)
> I'm trying to suggest to use 6 PC's =). I like the idea of threaded
> pool, but unfortunately i'm just a beginner in multihreading. Do you
> have other references besides W. Richard Steven's books?
The networking book from Stevens that I have, teaches some nonsense
about threading (using some errno like variable for error indication for
example). I can really recommend "Programming with POSIX Threads" from
David Butenhof.
How about following design:
1 thread waiting on the tcp listen port. For every incoming request a
new data structur is created with a buffer for the incoming request and
the socket. That structure is placed into a queue.
10 worker threads pick up to 1600/10 data structures from the queue and
pass the buffers to select. When ever a worker thread is returning from
select, it will check if the request where read completely from the
network and if so, do the data base query and reply to the request.
If the data base query might block for very long, you could pass the
data structure to an other queue followed by an other group of threads.
The same applies in case that the replies might block for longer.
regards
Torsten
Markus Elfring
> implement it.
Would you like to reuse anything from existing implementations?
http://httpd.apache.org/docs/2.2/mod/event.html
http://en.wikipedia.org/wiki/Thread_pool_pattern
> pthread_mutex_lock(&mutex2); // lock
> freeIdx2=GetFreeConnection(); // get free database connector
> selectdb(&myconn[freeIdx2], queryString, strlen(queryString), &res_set);
> pthread_mutex_unlock(&mutex2);
Is this source code section locked longer than it is really required for
efficient data processing?
Would you like to complete the checking of return values?
> sleep(1); // sleep 1 second
I suggest to replace this instruction by proper use of condition variables.
Regards,
Markus
Sonny
> If you have a limited number of database connection, I would expect the
> call to GetFreeConnection() to block until there is a database
> connection available. Holding a mutex around a blocking function is
> seldom a good idea and btw.: bad mutex name.
>
setup has 10 db connections. It scans until it finds one, going back
to the 1st index after it reaches the limit. I have a static counter
that marks the currently used connection so that the next to it will
be the one to look upon next time it is invoked. So this design is a
bad thing?
>
> How about following design:
> 1 thread waiting on the tcp listen port. For every incoming request a
> new data structur is created with a buffer for the incoming request and
> the socket. That structure is placed into a queue.
>
> 10 worker threads pick up to 1600/10 data structures from the queue and
> pass the buffers to select. When ever a worker thread is returning from
> select, it will check if the request where read completely from the
> network and if so, do the data base query and reply to the request.
>
> If the data base query might block for very long, you could pass the
> data structure to an other queue followed by an other group of threads.
> The same applies in case that the replies might block for longer.
>
Would you please give a skeleton code for this one, especially the
part with 10 worker threads, please =)
>>
>> pthread_mutex_lock(&mutex2); // lock
>> freeIdx2=GetFreeConnection(); // get free database connector
>> selectdb(&myconn[freeIdx2], queryString, strlen(queryString), &res_set);
>> pthread_mutex_unlock(&mutex2);
>>
>
>Is this source code section locked longer than it is really required for
>efficient data processing?
>Would you like to complete the checking of return values?
>
I locked this part because before 2 different threads are querying for
the same id in the database.
I'll try to explain the current program structure:
The server listens for new connections/messages
A new thread will be created every time a new client connects.
In the thread function:
The thread contains the message. It will be parsed using strtok_r and
put in a database, particularly in 2 tables. There is mutex lock when
getting the free connection and inserting it because i need the
mysql_insertid for the 2nd table:
pthread_mutex_lock(&mutex1); // lock
freeIdx1=GetFreeConnection(); // get free database connector
selectdb(&myconn[freeIdx1], queryString, strlen(queryString),
&res_set); // insert into table 1 the values
pthread_mutex_unlock(&mutex1);
insert_id = mysql_insert_id(myconn[freeIdx1].con);
sprintf(querystring, " insert ... values ( ..., %s)", ...,
insert_id);
selectdb(&myconn[freeIdx2], queryString, strlen(queryString),
&res_set); // insert into table 2 the values
After insertion, there will be another query, the one i posted before.
I'm checking if the value in table 1 is deleted (another program deals
with the deletion in a matter of seconds). If it is deleted, it has
been processed, then the functions will be called to query table 2
for the updates. This is where the problem using threads usually
occur. I mean, about 300(should be 1600) threads querying the database
in this part.
sprintf(queryString, "select ... where id=%s", insert_id);
for (;;)
{
pthread_mutex_lock(&mutex2); // lock
freeIdx2=GetFreeConnection(); // get free database
connector
selectdb(&myconn[freeIdx2], queryString,
strlen(queryString), &res_set);
pthread_mutex_unlock(&mutex2);
if (!res_set)
{ perror("Query error"); }
else if ( (numrow=mysql_num_rows(res_set)) == 0)
{
if ( strcmp(cpdata.actionID,"04") == 0) //
just checking for a flag
{
if (function1(sendBuffer, insert_id));
}
else
{
if (function2(sendBuffer, insert_id));
}
break;
}
else
sleep(1); // sleep for 1 second before
querying again
mysql_free_result(res_set); // i'm freeing the
result set so it won't clogged
}
Then the content/update will be written in the sendBuffer which will
be returned to client which send the message.
Then close the client connection.
This is actually the scenario. Take note that receiving/parsing/
inserting to database/retrieving data/sending back is inside a
function so that there will be no problems with the socket descriptor
for receiving and sending.
Btw the way, there will be a problem if the other program died (the
one who handles the deletion), the loop will not stop. I'm thinking of
using a counter for connection timeout, but since it is in a thread,
other threads might change this value.
I'm thinking if i use a threaded pool, how will it know the socket
descriptor when its going to reply back? TSD? How do I use it?
Actually in function 1, there is another inserting/waiting part like
before. Create another worker thread?
Well, its very simple using a thread per client approach and it works
for about 300 clients. Comments/Suggestions/Reactions are very
welcome. I understood the concepts on multithreading but having
problems applying it in a program. I need a simple example using a
threaded pool dealing with sockets.
>Would you like to reuse anything from existing implementations?
>http://httpd.apache.org/docs/2.2/mod/event.html
>http://en.wikipedia.org/wiki/Thread_pool_pattern
will be checking on it =)
Thanks for all of your patience. I'm really a beginner with this kind
of programming. Hopefully I won't reach your limit =)
Torsten Robitzki
>>If you have a limited number of database connection, I would expect the
>>call to GetFreeConnection() to block until there is a database
>>connection available. Holding a mutex around a blocking function is
>>seldom a good idea and btw.: bad mutex name.
>>
>
> Yup, I have a function that looks for a free database connection. My
> setup has 10 db connections. It scans until it finds one, going back
> to the 1st index after it reaches the limit. I have a static counter
> that marks the currently used connection so that the next to it will
> be the one to look upon next time it is invoked. So this design is a
> bad thing?
I can't see how this should work. How do you ensure that the next
connection isn't still in use. The usual pattern would be to have a
dynamic list of connections. When a thread requests a connection, the
first connection from the list is handed to the requesting thread and
than removed from the list. When the thread is done with the connection,
it will be inserted into the list again. If there is no connection in
the list, the requesting thread will be blocked until an other thread
frees a connection.
>
>
>
>>How about following design:
>>1 thread waiting on the tcp listen port. For every incoming request a
>>new data structur is created with a buffer for the incoming request and
>>the socket. That structure is placed into a queue.
>>
>>10 worker threads pick up to 1600/10 data structures from the queue and
>>pass the buffers to select. When ever a worker thread is returning from
>>select, it will check if the request where read completely from the
>>network and if so, do the data base query and reply to the request.
>>
>>If the data base query might block for very long, you could pass the
>>data structure to an other queue followed by an other group of threads.
>>The same applies in case that the replies might block for longer.
>>
>
> Would you please give a skeleton code for this one, especially the
> part with 10 worker threads, please =)
Every worker thread executes the same code:
for (;;) {
list open_connections;
open_connections.add(queue.get_at_maximum(160));
select(open_connections);
for ( list::iterator i = open_connections.begin();
i != open_connections.end(); ++i )
{
if ( i->read_all() ) {
db_connection connect;
i->db_query(connect);
i->send_response();
}
}
}
> This is actually the scenario. Take note that receiving/parsing/
> inserting to database/retrieving data/sending back is inside a
> function so that there will be no problems with the socket descriptor
> for receiving and sending.
Ok, than you actual problem is that you have a really broken design.
Polling the database is your problem. Can't you change the IPC to use
pipes or something more sane?
If not, you could dedicate a single thread to that polling and create a
sql statement that polls for all outstanding deletions.
> Btw the way, there will be a problem if the other program died (the
> one who handles the deletion), the loop will not stop. I'm thinking of
> using a counter for connection timeout, but since it is in a thread,
> other threads might change this value.
Keep the starting time for every connection when you started to poll for
the db.
> I'm thinking if i use a threaded pool, how will it know the socket
> descriptor when its going to reply back?
You should really read some basics about thread pools. There is nothing
magic about them. It's just a bunch of threads usually all executing the
same function and connected to other pools or threads by queues.
> TSD? How do I use it?
No, use a queue to pass data to the pool.
> Actually in function 1, there is another inserting/waiting part like
> before. Create another worker thread?
> Well, its very simple using a thread per client approach and it works
> for about 300 clients. Comments/Suggestions/Reactions are very
> welcome. I understood the concepts on multithreading but having
> problems applying it in a program. I need a simple example using a
> threaded pool dealing with sockets.
The sockets aren't your problem. Your are polling the database with 300
threads, that might be your problem.
> Thanks for all of your patience. I'm really a beginner with this kind
> of programming. Hopefully I won't reach your limit =)
You should buy a good book and read up some basics :-)
kind regards,
Torsten
llothar
> should be capable of handling about 1600 concurrent connections. It
> was tested on a pc with 256MB RAM and no major problems were
> encountered if there were about 300 clients. However beyond 300 I'm
> experiencing trouble creating (p)threads, (cannot allocate memory). I
Despite what other people say. Something like 2000 threads shouldn't
be
a problem for a good OS, restrict the stack to 256KByte and you have
512MB
data area for this. The only fucked up system who can't handle this is
Linux with
its complete stupid and insane linux-threads implementation.
If it is possible to connect 2000 clients to a DBMS is another problem
not
on-topic in this group.
Sonny
> Sonny wrote:
> >>If you have a limited number of database connection, I would expect the
> >>call to GetFreeConnection() to block until there is a database
> >>connection available. Holding a mutex around a blocking function is
> >>seldom a good idea and btw.: bad mutex name.
>
> > Yup, I have a function that looks for a free database connection. My
> > setup has 10 db connections. It scans until it finds one, going back
> > to the 1st index after it reaches the limit. I have a static counter
> > that marks the currently used connection so that the next to it will
> > be the one to look upon next time it is invoked. So this design is a
> > bad thing?
>
> I can't see how this should work. How do you ensure that the next
> connection isn't still in use. The usual pattern would be to have a
> dynamic list of connections. When a thread requests a connection, the
> first connection from the list is handed to the requesting thread and
> than removed from the list. When the thread is done with the connection,
> it will be inserted into the list again. If there is no connection in
> the list, the requesting thread will be blocked until an other thread
> frees a connection.
>
flag. When a connection will be used for querying, it will be mark as
1, after finishing it will be marked as 0
typedef struct
{
MYSQL *conn;
int flg;
} M_DB_CONN;
M_DB_CONN myconn[NUMDBCONNECTION];
/* Check for free database connector */
int GetFreeConnection(void)
{
static int freex = 0;
int y=0;
while (1)
{
if(myconn[freex].flg==0)
{
myconn[freex].flg=1;
y=freex++;
if (freex >= NUMDBCONNECTION) freex=0;
return y;
}
else
freex++;
if (freex >= NUMDBCONNECTION) freex=0;
}
}
>
>
> >>How about following design:
> >>1 thread waiting on the tcp listen port. For every incoming request a
> >>new data structur is created with a buffer for the incoming request and
> >>the socket. That structure is placed into a queue.
>
> >>10 worker threads pick up to 1600/10 data structures from the queue and
> >>pass the buffers to select. When ever a worker thread is returning from
> >>select, it will check if the request where read completely from the
> >>network and if so, do the data base query and reply to the request.
>
> >>If the data base query might block for very long, you could pass the
> >>data structure to an other queue followed by an other group of threads.
> >>The same applies in case that the replies might block for longer.
>
> > Would you please give a skeleton code for this one, especially the
> > part with 10 worker threads, please =)
>
> Every worker thread executes the same code:
>
> for (;;) {
> list open_connections;
> open_connections.add(queue.get_at_maximum(160));
>
> select(open_connections);
> for ( list::iterator i = open_connections.begin();
> i != open_connections.end(); ++i )
> {
> if ( i->read_all() ) {
> db_connection connect;
> i->db_query(connect);
> i->send_response();
> }
> }
>
> }
could you elaborate more. Is list a data structure or an enumeration
of the list. I'm not good with the C++ syntax
What would you suggest? The data to be retrieved is in the database.
The signal that there is an update is when the entry is deleted in the
table. So basically, i have to poll it.
Chris Thomasson
news:1188518929.9...@e9g2000prf.googlegroups.com...
> On 30 Aug., 14:26, Sonny <smani...@gmail.com> wrote:
>> should be capable of handling about 1600 concurrent connections. It
>> was tested on a pc with 256MB RAM and no major problems were
>> encountered if there were about 300 clients. However beyond 300 I'm
>> experiencing trouble creating (p)threads, (cannot allocate memory). I
>
> Despite what other people say. Something like 2000 threads shouldn't
> be
> a problem for a good OS, restrict the stack to 256KByte and you have
> 512MB
> data area for this. The only fucked up system who can't handle this is
> Linux with
> its complete stupid and insane linux-threads implementation.
[...]
2,000 threads on a 2,000+ cpu system right? You should really try to have no
more than 2-3 kernel threads per-processor...
Torsten Robitzki
> I used a structure for the database connection, a MySQL type and a
> flag. When a connection will be used for querying, it will be mark as
> 1, after finishing it will be marked as 0
How do you think a thread that owns a connection should finnish its work
when the waiting threads waist the cpu time by spinning? You should read
the chapter about condition variables in the book you should get your
self. You can't write a reasonable, portable multi threaded program
without condition variables. So you have to learn it anyway.
> could you elaborate more. Is list a data structure or an enumeration
> of the list. I'm not good with the C++ syntax
list is a class that implements a list in this example. And I'm not good
with C :-)
> What would you suggest? The data to be retrieved is in the database.
> The signal that there is an update is when the entry is deleted in the
> table. So basically, i have to poll it.
Well, in my last post I suggested to dedicate only _one_ thread to poll
the data base and to query all outstanding responses at once. Or even
better to use pipes for IPC, not the data base.
kind regards,
Torsten
David Schwartz
> /* Check for free database connector */
> int GetFreeConnection(void)
> {
>
> static int freex = 0;
> int y=0;
> while (1)
> {
> if(myconn[freex].flg==0)
> {
> myconn[freex].flg=1;
> y=freex++;
> if (freex >= NUMDBCONNECTION) freex=0;
> return y;
> }
> else
> freex++;
> if (freex >= NUMDBCONNECTION) freex=0;
> }
>
>
> }
This obviously can't work. Either the thread running those code holds
a mutex to protect the table of free connections or it doesn't. If it
does, then how can another thread mark a connection free? If all
connections are in use, it will spin forever. If it doesn't, then what
protects this structure from simultaneous access?
You also didn't show how any thread marks that it is no longer using a
connection. So there could be problems there as well.
Using a thread-per-client approach, you are starting out with several
strikes against you. So you had better get everything else exactly
right.
DS
Sonny
> On Aug 30, 5:42 pm, Sonny <smani...@gmail.com> wrote:
>
>
>
>
>
> > /* Check for free database connector */
> > int GetFreeConnection(void)
> > {
>
> > static int freex = 0;
> > int y=0;
> > while (1)
> > {
> > if(myconn[freex].flg==0)
> > {
> > myconn[freex].flg=1;
> > y=freex++;
> > if (freex >= NUMDBCONNECTION) freex=0;
> > return y;
> > }
> > else
> > freex++;
> > if (freex >= NUMDBCONNECTION) freex=0;
> > }
>
> > }
>
> This obviously can't work. Either the thread running those code holds
> a mutex to protect the table of free connections or it doesn't. If it
> does, then how can another thread mark a connection free? If all
> connections are in use, it will spin forever. If it doesn't, then what
> protects this structure from simultaneous access?
>
> You also didn't show how any thread marks that it is no longer using a
> connection. So there could be problems there as well.
>
int selectdb(M_DB_CONN *con, char *query, unsigned int len, MYSQL_RES
**res1)
{
if ( !(mysql_real_query(con->conn, query, len)) )
{
*res1 = mysql_store_result(con->conn);
con->flg=0;
return TRUE;
}
con->flg=0;
printf("%s: %u %s\n", query, mysql_errno(con->conn),
mysql_error(con->conn) );
perror("selectdb error ");
return FALSE;
}
void processdb(M_DB_CONN *con, CONFIGDATA cfg, char *query, unsigned
int len)
{
if (mysql_real_query(con->conn, query, len)) {
printf("%s: %s\n", query, mysql_error(con->conn) );
perror("processdb error");
}
con->flg=0;
}
Markus Elfring
> My setup has 10 db connections.
Would you like to try a database implementation that supports concurrent queries
on the same connection?
http://dev.mysql.com/doc/refman/5.1/en/threaded-clients.html
Are you interested to know the performance characteristics if the corresponding
identifiers will be managed in two lists (CIDs in use and ready for reuse)?
Regards,
Markus
Sonny
> Well, in my last post I suggested to dedicate only _one_ thread to poll
> the data base and to query all outstanding responses at once. Or even
> better to use pipes for IPC, not the data base.
>
> kind regards,
> Torsten
query for the all ID's deleted or check one by one the IDs using one
thread?
Btw, If anyone interested with the code, I'll email it. It has 7 files
=)
Torsten Robitzki
> btw, what do you mean by outstanding responses/deletion?
If I got you right, you are polling for an other process deleting a row
in the database.
> One big sql
> query for the all ID's deleted or check one by one the IDs using one
> thread?
If you have inserted 10 rows in the data base and you are waiting for
this 10 rows to become deleted, don't ask the data base 10 times.
SELECT id FROM table WHERE id IN (3, 56, 43, 12... );
> Btw, If anyone interested with the code, I'll email it. It has 7 files
> =)
Sorry, I have anough work to do. Just hire a decent consultant that do
the work for you.
best regards,
Torsten
Chris Friesen
> The only fucked up system who can't handle this is
> Linux with
> its complete stupid and insane linux-threads implementation.
Have you actually tried a moderately recent linux distro? Except for a
few oddball architectures, they've all switched over to NPTL which is
perfectly capable of handling thousands of threads.
Chris
Sonny
> > btw, what do you mean by outstanding responses/deletion?
>
> If I got you right, you are polling for an other process deleting a row
> in the database.
updating the database
>
> > One big sql
>
> > query for the all ID's deleted or check one by one the IDs using one
> > thread?
>
> If you have inserted 10 rows in the data base and you are waiting for
> this 10 rows to become deleted, don't ask the data base 10 times.
>
> SELECT id FROM table WHERE id IN (3, 56, 43, 12... );
>
i see. looks like its complicated (generating the list in the "IN ( )
part). Plus the select statement will show the IDs that are not still
processed where the one that I need are those not shown
> > Btw, If anyone interested with the code, I'll email it. It has 7 files
> > =)
>
> Sorry, I have anough work to do. Just hire a decent consultant that do
> the work for you.
>
just a wishful thinking =D
> Would you like to try a database implementation that supports concurrent queries
> on the same connection?
>http://dev.mysql.com/doc/refman/5.1/en/threaded-clients.html
actually i have read that before, gonna check with the sys ad about
that.
>
>Are you interested to know the performance characteristics if the corresponding
>identifiers will be managed in two lists (CIDs in use and ready for reuse)?
What do you mean by CIDs?
Anyway thanks again guys for your patience. It's very hard to find a
book for that this topic here in our country (RP), I can only find
them in online bookstores from the US. Unfortunately, I have no credit
cards. Would you recommend an e-book, besides Beej's? Or a good web
site? I tried to google it but most searches are lame.
Markus Elfring
Connection identifiers - It seems that pooling will be required for concurrent
queries to achieve complete thread-safety with MySQL.
> Or a good web site?
Examples:
http://www.codeproject.com/index.asp
http://monkey.org/~provos/libevent/
http://www.hoard.org/
http://quickserver.org/
http://www.orbzone.org/
http://en.wikipedia.org/wiki/Thread_%28computer_science%29
> I tried to google it but most searches are lame.
That's strange - Which information sources are you missing?
Regards,
Markus
Ian Collins
Whether thousands of threads are supported in one application and
Whether thousands of threads are a good idea are two different questions.
--
Ian Collins.
Chris Thomasson
"Ian Collins" <ian-...@hotmail.com> wrote in message
news:5jrbkoF...@mid.individual.net...
> Chris Friesen wrote:
>> llothar wrote:
>>> The only fucked up system who can't handle this is
>>> Linux with
>>> its complete stupid and insane linux-threads implementation.
>>
>> Have you actually tried a moderately recent linux distro? Except for a
>> few oddball architectures, they've all switched over to NPTL which is
>> perfectly capable of handling thousands of threads.
>>
> Ever tried waking them all up at once :)
Ouch! That very dangerous because it would unleash a massive thundering
heard upon your poor unsuspecting application and promptly extinguish any
possibility of scalability.
> Whether thousands of threads are supported in one application and
> Whether thousands of threads are a good idea are two different questions.
Indeed.
Sonny
> > What do you mean by CIDs?
>
> Connection identifiers - It seems that pooling will be required for concurrent
> queries to achieve complete thread-safety with MySQL.
>
> > Or a good web site?
>
> Examples:http://www.codeproject.com/index.asp
>http://monkey.org/~provos/libevent/
>http://www.hoard.org/
>http://quickserver.org/
>http://www.orbzone.org/
>http://en.wikipedia.org/wiki/Thread_%28computer_science%29
I got some updates on what will be the final setup. The program will
be instantiated 16 times with 100 clients/threads each to have the
1600 connections, just using 16 diff ports. However testing 2-4
instances, the same problem still occur. I'm still checking with the
sys ad about the multithreaded connection in MySQL. So can we say
that the problem is with MySQL? Does the thread per client approach
can still be used, no change with the current program structure?
David Schwartz
> int selectdb(M_DB_CONN *con, char *query, unsigned int len, MYSQL_RES
> **res1)
> {
> if ( !(mysql_real_query(con->conn, query, len)) )
> {
> *res1 = mysql_store_result(con->conn);
> con->flg=0;
> return TRUE;
> }
> con->flg=0;
> printf("%s: %u %s\n", query, mysql_errno(con->conn),
> mysql_error(con->conn) );
> perror("selectdb error ");
> return FALSE;
>
}
Wow, this code makes no sense. What mutex protects "con->flg"? And if
this code holds that mutex, what's the point in having con->flg at
all?
DS
Sonny
the function doesn't have a mutex. i used mutex only twice. first is
when i got the index of the first open db connection since i need that
connection for the mysql_insert_id(). The second one is during the
infinite loop. What would you recommend?
David Schwartz
> > Wow, this code makes no sense. What mutex protects "con->flg"? And if
> > this code holds that mutex, what's the point in having con->flg at
> > all?
> the function doesn't have a mutex.
Then what protects con->flg from concurrent access?
> i used mutex only twice. first is
> when i got the index of the first open db connection since i need that
> connection for the mysql_insert_id(). The second one is during the
> infinite loop.
Okay, I don't follow. Does that mutex protect accesses to con->flg or
not? If so, why is there no mutex lock in the function you pasted? If
not, why do you need to hold it when you get the index of the first
open db connection?
> What would you recommend?
Protecting the table of connections with a mutex and holding the mutex
only when accessing that table. You must release the mutex when you
need to wait for a connection so that other threads can release the
connections they are using! Condition variables make this simple
(because they offer an atomic 'unlock and wait' function).
DS
Markus Elfring
Would you like to clarify this system issue of concurrent queries with help from
the forum "comp.databases.mysql"?
> So can we say that the problem is with MySQL?
Do you consider to compare the performance and reliability of different
databases for your application?
> Does the thread per client approach can still be used, no change with the current program structure?
It depends - Would you like to invest in bigger computing hardware if the
software was not implemented as efficient as more experienced developers would
expect? When are you going to reuse process, thread and connection pool
implementations like it is demonstrated by the Apache 2.2 web server?
Regards,
Markus
Markus Elfring
> int GetFreeConnection(void)
How much can the explanations from the document "Connection Pooling in Apache"
help in your software?
http://www.apachetutor.org/dev/reslist
Regards,
Markus
Sonny
Thanks for all your input guys. Just busy with some other things. I'll
be looking at it and see what will happen. I'll update you soon.
Still, feel free to add comments/suggestions. They're very much
welcome! =)
Sonny
mysql_store_result, and somehow, i don't encounter problems. I used 4
instances with 600 threads and it is still working.
Another news, we're gonna change the architecture. The program will be
divided, one that listens and inserts into the db, and another one
that checks for updates and sends it back. So I guess I'll see you in
the next thread. Again thanks for your advices. Additional advices
will still be accepted.
llothar
> 2,000 threads on a 2,000+ cpu system right? You should really try to have no
> more than 2-3 kernel threads per-processor...
All modern Kernels should have O(1) time for scheduling etc. and 0
overhead for
suspended threads.
Restricting the number of active threads is pretty easy, just setup
a semaphore with a positive count.
Using threads often gives you a much much easier design and today this
is
more important then using buggy complicated to write state machines.
Markus Elfring
> mysql_store_result, and somehow, i don't encounter problems.
Do you use a kind of global lock for every request now?
Are you sure that all data accesses are synchronised with proper granularity in
your source code?
Regards,
Markus
Sonny
This is the new struct:
typedef struct
{
MYSQL *conn;
int flg;
pthread_mutex_t lock;
} M_DB_CONN;
Basically in my function that query the dbase, i put a mutex lock
before mysql_real_q and unlock it after mysql_store_result.
int selectdb(M_DB_CONN *con, char *query, unsigned int len, MYSQL_RES
**res1)
{
pthread_mutex_lock(&con->lock);
if ( !(mysql_real_query(con->conn, query, len)) )
{
*res1 = mysql_store_result(con->conn);
pthread_mutex_unlock(&con->lock);
con->flg=0;
return TRUE;
}
pthread_mutex_unlock(&con->lock);
con->flg=0;
printf("%s: %u %s\n", query, mysql_errno(con->conn),
mysql_error(con->conn) );
perror("selectdb error ");
return FALSE;
}
Anyway, the new model is going to be a select server and one dbase
connection. There will be 2 programs, one the listens for the message,
and one that sends back the reply There will be 3 instances for each
program. The 1600 connections will be divided among them.
Markus Elfring
> {
> MYSQL *conn;
> int flg;
> pthread_mutex_t lock;
> } M_DB_CONN;
I suggest to delete the integer flag because it is not needed for mutual
exclusion. How do you think about to manage two lists for connections in use and
ready for reuse?
Did you improve your predicate handling with condition variables in other places
of your source code?
Regards,
Markus
Sonny
> > typedef struct
> > {
> > MYSQL *conn;
> > int flg;
> > pthread_mutex_t lock;
> > } M_DB_CONN;
>
> I suggest to delete the integer flag because it is not needed for mutual
> exclusion. How do you think about to manage two lists for connections in use and
> ready for reuse?
Yup, that's what i was thinking
> Did you improve your predicate handling with condition variables in other places
> of your source code?
>
> Regards,
> Markus
The model was changed, it is now composed of 2 program, one that
listens and inserts in the db and the other one checks for updates and
sends it. And each one has 1 db connections. I am now using a
select() server. The 2 programs will have 3 instances each.
Markus Elfring
> listens and inserts in the db and the other one checks for updates and
> sends it. And each one has 1 db connections. I am now using a
> select() server. The 2 programs will have 3 instances each.
Do they use a more efficient architecture than the approach "thread per
connection" now?
http://zeroc.com/faq/threadPerConnection.html
http://omniorb.sourceforge.net/omni41/omniORB/omniORB008.html#toc40
http://today.java.net/pub/a/today/2007/02/13/architecture-of-highly-scalable-nio-server.html
Is there any inter-process communication involved between them?
Regards,
Markus
Sonny
> Do they use a more efficient architecture than the approach "thread per
>
it just one process for both
> Is there any inter-process communication involved between them?
>
none. each client just sends a message and gets a response
Markus Elfring
That does not say anything about the performance potential of your adjusted
software structure. How much are you concerned about communication efficiency?
Do you accept the explanations that were given by Chris Pearson in the
discussion "Stay away of creating thread-per-connection servers"?
http://groups.google.de/group/alt.winsock.programming/msg/f1447c0fdfef04f6
Are you used to consequences by constraints on limited system resources of
operating systems?
Regards,
Markus
Sonny
> > it just one process for both
>
> That does not say anything about the performance potential of your adjusted
> software structure. How much are you concerned about communication efficiency?
program is still being used. I can say that the current model is
already tried and tested, though I can say the one with the threads
are faster.
> Do you accept the explanations that were given by Chris Pearson in the
> discussion "Stay away of creating thread-per-connection servers"?>http://groups.google.de/group/alt.winsock.programming/msg/f1447c0fdfe...
Actually i don't understand some of the technical details. But it
actually depends, threads per connection is a simpler design and if
the processes are not that intensive, maybe for a small project. But
for scalabilty, not good.
> Are you used to consequences by constraints on limited system resources of
> operating systems?
uhm, not yet =)
Markus Elfring
> actually depends, threads per connection is a simpler design and if
> the processes are not that intensive, maybe for a small project. But
> for scalabilty, not good.
Would you like to improve the performance, scalability and stability of your
software to get more power for your application?
How do you think about to apply more well-known ideas for highly concurrent
server applications to the current development?
http://www.eecs.harvard.edu/~mdw/proj/seda/
http://proactive.inria.fr/
http://terracotta.org/
Regards,
Markus
Sonny
> > Actually i don't understand some of the technical details. But it
> > actually depends, threads per connection is a simpler design and if
> > the processes are not that intensive, maybe for a small project. But
> > for scalabilty, not good.
>
> Would you like to improve the performance, scalability and stability of your
> software to get more power for your application?
>
transition stage of the whole system. It will be used for only about
a month =(
> How do you think about to apply more well-known ideas for highly concurrent
> server applications to the current development?http://www.eecs.harvard.edu/~mdw/proj/seda/http://proactive.inria.fr/http://terracotta.org/
>
> Regards,
> Markus
Maybe for future projects. Thanks sir for the info.