Q4M throughput tuning

[James McGill]

I've been analyzing our Q4M throughput and have found that our install is not nearly as fast as indicated at http://labs.cybozu.co.jp/blog/kazuhoatwork/2008/06/q4m_06_release_and_benchmarks.php .

I'm curious what type of throughput per server others are getting with Q4M and if anybody has some recommendations on where to begin debugging/tuning our setup?

VAR_LENGTH=512 MESSAGES=1000 CONCURRENCY=10 DBI='dbi:mysql:test;mysql_socket=/data/Q4M_DB/data/mysql3306.sock;user=root' t/05-multirw.t 

1..4

ok 1 - check number of messages

ok 2 - min value of received message

ok 3 - max value of received message

ok 4 - should have no rows in table

Multi-reader-writer benchmark result:

    Number of messages: 1000

    Number of readers:  10

    Elapsed:            9.116 seconds

    Throughput:         109.695 mess./sec.

We're using pre-built binaries of Q4M 0.9.1 for x86_64 on CentOS.

cheers,

james

[Kazuho Oku]

Hi,

The thoughput would become higher as you increase concurrency.

The main bottleneck of Q4M is generally the disk synchronization IOPS.
I would have to say that in your case the disk seems to be a bit
slow, but the throughput should become higher in proportion to the
number of workers (if you are handling small messages).

2010/3/16 James McGill <jbmc...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "Q4M - a Message Queue for MySQL" group.
> To post to this group, send email to q4m-g...@googlegroups.com.
> To unsubscribe from this group, send email to
> q4m-general...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/q4m-general?hl=en.
>

--
Kazuho Oku

[James McGill]

hi,

We determined that the raid controller we ran the benchmarks on did
not have a battery backed write cache.

We installed a BBWC upgrade on the server and performance is really
great when running the benchmark script now:

concurrency 10 we get about 4500 messages / sec
concurrency 100 we get about 7000 messages / sec
concurrency 300 we get about 20000 messages / sec

If anybody is not getting high volumes of concurrent throughput
through Q4M be sure to check that you have write cache enabled.

cheers,
james