Messages rates are always 0, RabbitMQ 3.6.5 M1

282 views
Skip to first unread message

zhenbo zeng

unread,
Aug 4, 2016, 3:22:07 AM8/4/16
to rabbitmq-users
I meet the problem like this discussion https://groups.google.com/forum/#!topic/rabbitmq-users/0Ne1NvZCNww and issue#266, with v3.6.3

And then I try the new tag 3.6.5 M1, download rabbitmq-server-generic-unix-3.6.4.901.tar.xz from https://github.com/rabbitmq/rabbitmq-server/releases/tag/rabbitmq_v3_6_5_milestone1

The problem still exist(overview, exchanges, queues pages), and I can reproduce this problem with my script.


receive_logs.erl
emit_log.erl

Diana Parra Corbacho

unread,
Aug 4, 2016, 3:30:05 AM8/4/16
to rabbitm...@googlegroups.com
Hi Zhenbo,

I will give that a try.

Regards,
Diana

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Diana Parra Corbacho

unread,
Aug 4, 2016, 5:01:53 AM8/4/16
to rabbitm...@googlegroups.com
Hi Zhenbo,

I tested the release with your script and it shows correctly the stats.

You are getting a real zero there, as you can see in the graphs the rate drops to zero at some point. That might be the end of your test or even that the system becomes unresponsive under load. I guess you are using big machines to run that, my laptop certainly can't cope with it.

Regards,
Diana
Screen Shot 2016-08-04 at 9.55.14 AM.png

zhenbo zeng

unread,
Aug 4, 2016, 5:11:06 AM8/4/16
to rabbitmq-users
8 CPUs 16G Mem
My test machine.

在 2016年8月4日星期四 UTC+8下午5:01:53,Diana Corbacho写道:

zhenbo zeng

unread,
Aug 4, 2016, 5:28:21 AM8/4/16
to rabbitmq-users

As you can see the follow graphs, at the time 17:19:00, message rates are  0 in the chart of G1, but non-zero in the chart of G2,G3,G4

G1:



G2:


G3:


G4:




zhenbo zeng

unread,
Aug 4, 2016, 5:36:05 AM8/4/16
to rabbitmq-users

The CPU and memory usage are normal.




在 2016年8月4日星期四 UTC+8下午5:01:53,Diana Corbacho写道:
Hi Zhenbo,

zhenbo zeng

unread,
Aug 4, 2016, 5:36:51 AM8/4/16
to rabbitmq-users

The CPU and memory usage are normal.




在 2016年8月4日星期四 UTC+8下午5:01:53,Diana Corbacho写道:
Hi Zhenbo,

Diana Parra Corbacho

unread,
Aug 4, 2016, 5:40:28 AM8/4/16
to rabbitm...@googlegroups.com
Hi Zhenbo,

I can see in all the graphs that you pasted that the instant rates are 0, but that is normal as the rate has just dropped. Don't you have non-zero rates before?

If you are looking for the rates for the whole interval, you should select the "moving average" view. See the difference in the attached screenshots.




​Kind regards,
Diana

zhenbo zeng

unread,
Aug 4, 2016, 6:01:35 AM8/4/16
to rabbitmq-users
The "moving average" view is non-zero, the "chart" view and "current value" view are zero.

We closed the the display of message rates before 3.6.x, when there are so much consumers queues.

The management UI shows some warning message with 3.5.7, so we set the rates_mode to none

The management statistics database currently has a queue of 2198 events to process. If this number keeps increasing, so will the memory used by the management plugin. You may find it useful to set the rates_mode config item to none.

在 2016年8月4日星期四 UTC+8下午5:40:28,Diana Corbacho写道:

Diana Parra Corbacho

unread,
Aug 4, 2016, 6:11:27 AM8/4/16
to rabbitm...@googlegroups.com
I do not see any problem with the stats, the instant rate is zero because events might still be arriving with the actual data.
The moving average is non-zero because that covers the whole last minute that you're querying, not only the last 5 seconds.

As you see, rates are corrected soon afterwards. If the system is busy, it might take slightly longer to process events.

zhenbo zeng

unread,
Aug 4, 2016, 6:26:23 AM8/4/16
to rabbitmq-users
We want to monitor the current message rates to check whether the system work normal or not.
If the current message rates are zero, we might have to try another way to monitor the system.

在 2016年8月4日星期四 UTC+8下午6:11:27,Diana Corbacho写道:

Michael Klishin

unread,
Aug 4, 2016, 9:47:11 AM8/4/16
to rabbitm...@googlegroups.com
They *can* be zero for a moment in time. Making monitoring decisions on just one
such sample is guaranteed to trigger false positives. Using a weighted average
of some kind is probably a better idea.
MK

Staff Software Engineer, Pivotal/RabbitMQ

zhenbo zeng

unread,
Aug 4, 2016, 10:33:39 PM8/4/16
to rabbitmq-users
Thanks Michael and Diana

In Diana's test, small amount of queues and consumes, the message rates displayed on the mgmt UI look normal(non-zero).
And in my test, large amount of queues and consumers, with low publish messages rates, the message rates displayed on the mgmt UI always 0.
Then, I run my test again, just with more higher publish messages rates, the message rates almost are non-zero.

What make me confused is the results of the mgmt UI are different when work with different workload.



在 2016年8月4日星期四 UTC+8下午9:47:11,Michael Klishin写道:
Reply all
Reply to author
Forward
0 new messages