RabbitMQ perfomance test

[Andrey Novikov]

Hi, all!

Help me understand the results of the performance test.

Results I am getting while running performance tests on RabbitMQ I cannot explain.

Using standard tool "com.rabbitmq.perf.PerfTest" I ran two simple commands, in which the only difference is the packet size (see screenshot).

I expected to get the lower latency for the smaller packet size but output shows that the large packet is the shorter latency is. Is this possible? What am I doing wrong?

[Michael Klishin]

There are many factors that affect latency. PerfTest has a reasonably detailed guide

about the options it supports, various workloads and test scenarios.

100 bytes vs. 500 bytes is hardly a significant difference in message size and a single

short run is not statistically significant.

Try 100 bytes vs 5 megabytes or so, 10 or more runs (run results can be exported to CSV

for easier aggregation) and longer runs (several minutes or more if you are willing to wait).

It's also worth mentioning that 1 publisher and 1 consumer will collectively use 2-3 cores [2].

1. http://www.rabbitmq.com/java-tools.html

2. http://www.rabbitmq.com/queues.html#runtime-characteristics

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Andrey Novikov]

On your advice, I conducted two more tests, where the message size was 100 bytes and 1 MB (5 MB does not pull). If again look at the median latency, then at 100 bytes it is still greater than when the message size is 1 MB. However, as the message size increases, the bandwidth drops (everything is fine here). Even with such a simple example, I want to understand why this happens. I attach the file with the results.

суббота, 9 июня 2018 г., 17:13:47 UTC+3 пользователь Michael Klishin написал:

There are many factors that affect latency. PerfTest has a reasonably detailed guide

about the options it supports, various workloads and test scenarios.

100 bytes vs. 500 bytes is hardly a significant difference in message size and a single

short run is not statistically significant.

Try 100 bytes vs 5 megabytes or so, 10 or more runs (run results can be exported to CSV

for easier aggregation) and longer runs (several minutes or more if you are willing to wait).

It's also worth mentioning that 1 publisher and 1 consumer will collectively use 2-3 cores [2].

1. http://www.rabbitmq.com/java-tools.html

2. http://www.rabbitmq.com/queues.html#runtime-characteristics

On Sat, Jun 9, 2018 at 3:49 PM, Andrey Novikov <andrew.no...@yandex.ru> wrote:

Hi, all!

Help me understand the results of the performance test.

Results I am getting while running performance tests on RabbitMQ I cannot explain.

Using standard tool "com.rabbitmq.perf.PerfTest" I ran two simple commands, in which the only difference is the packet size (see screenshot).

I expected to get the lower latency for the smaller packet size but output shows that the large packet is the shorter latency is. Is this possible? What am I doing wrong?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Katie Holly]

Hi,

try with the options listed here: https://www.rabbitmq.com/networking.html#tuning-for-throughput-tcp-buffers

Take a look at the "nodelay" option which is responsible for making sure that TCP packets don't get buffered up when the buffer doesn't reach a certain size - This option is disabled by default in the Linux kernel and you should leave it disabled with high traffic applications like a busy RabbitMQ server.

Best regards

Katie Holly

> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com <mailto:rabbitmq-user...@googlegroups.com>.
> To post to this group, send email to rabbitm...@googlegroups.com <mailto:rabbitm...@googlegroups.com>.

[Michael Klishin]

Most RabbitMQ clients and RabbitMQ itself disable Nagle's algorithm by default.

On Sat, Jun 9, 2018 at 9:04 PM, Katie Holly <jvc6...@meo.ws> wrote:

Hi,

try with the options listed here: https://www.rabbitmq.com/networking.html#tuning-for-throughput-tcp-buffers

Take a look at the "nodelay" option which is responsible for making sure that TCP packets don't get buffered up when the buffer doesn't reach a certain size - This option is disabled by default in the Linux kernel and you should leave it disabled with high traffic applications like a busy RabbitMQ server.

Best regards

Katie Holly

On 06/09/2018 02:49 PM, Andrey Novikov wrote:
> Hi, all!
>
> Help me understand the results of the performance test.
>
> Results I am getting while running performance tests on RabbitMQ I cannot explain.
>
> Using standard tool "com.rabbitmq.perf.PerfTest" I ran two simple commands, in which the only difference is the packet size (see screenshot).
>
> I expected to get the lower latency for the smaller packet size but output shows that the large packet is the shorter latency is. Is this possible? What am I doing wrong?
>
>
> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com <mailto:rabbitmq-users+unsub...@googlegroups.com>.
> To post to this group, send email to rabbitmq-users@googlegroups.com <mailto:rabbitmq-users@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send an email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

With PerfTest clients and TCP stack are involved not once but twice (on the publisher and consumer end),

so Wireshark [1] would reveal a lot of relevant information.

Collecting JVM and Erlang runtime metrics would be more involved. PerfTest GC pauses and context switching

behavior of the node (which is a metric exposed by the management plugin) are two good starting points.

1. http://www.rabbitmq.com/amqp-wireshark.html

On Sun, Jun 10, 2018 at 5:30 AM, Michael Klishin <mkli...@pivotal.io> wrote:

Most RabbitMQ clients and RabbitMQ itself disable Nagle's algorithm by default.

On Sat, Jun 9, 2018 at 9:04 PM, Katie Holly <jvc6...@meo.ws> wrote:

Hi,

try with the options listed here: https://www.rabbitmq.com/networking.html#tuning-for-throughput-tcp-buffers

Take a look at the "nodelay" option which is responsible for making sure that TCP packets don't get buffered up when the buffer doesn't reach a certain size - This option is disabled by default in the Linux kernel and you should leave it disabled with high traffic applications like a busy RabbitMQ server.

Best regards

Katie Holly

On 06/09/2018 02:49 PM, Andrey Novikov wrote:
> Hi, all!
>
> Help me understand the results of the performance test.
>
> Results I am getting while running performance tests on RabbitMQ I cannot explain.
>
> Using standard tool "com.rabbitmq.perf.PerfTest" I ran two simple commands, in which the only difference is the packet size (see screenshot).
>
> I expected to get the lower latency for the smaller packet size but output shows that the large packet is the shorter latency is. Is this possible? What am I doing wrong?
>
>
> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com <mailto:rabbitmq-users+unsubscri...@googlegroups.com>.
> To post to this group, send email to rabbitm...@googlegroups.com <mailto:rabbitmq-users@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.

To post to this group, send an email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Luke Bakken]

Hi Andrey,

Could you please let us know what RabbitMQ version, Erlang version, Java version and operating system you are using? Are you running PerfTest on the same machine as RabbitMQ? What is the network like? What is the MTU for the network?

I had the first same thought as Katie that this had something to do with the TCP nodelay option but we do try and disable that when RabbitMQ starts and in all clients like Michael said.

Thanks,

Luke

[Andrey Novikov]

RabbitMQ version 3.5.7
Erlang version 7.3

Java version 1.8.0

Linux Xubuntu 16.04
Yes, I running PerfTest on the same machine as RabbitMQ
Network 100 Mb/s
The value of MTU did not change, and probably is 1500

Hardware specification

CPU: 4x  Intel Core i5-2450M @ 2.50 GHz

RAM: 4GB

Storage: 500 GB

воскресенье, 10 июня 2018 г., 19:05:31 UTC+3 пользователь Luke Bakken написал:

[Luke Bakken]

Hi Andrey,

Both your RabbitMQ and Erlang version are quite out-of-date. Please see this guide for installing more recent versions:

https://www.rabbitmq.com/install-debian.html

Thanks,

Luke