Erlang config beam.smp CPU usage

1,666 views
Skip to first unread message

Chao Xu

unread,
May 3, 2017, 8:36:08 AM5/3/17
to rabbitmq-users
Beam.smp is causing very high CPU usage and impacting other applications. From other threads I heard that Erlange vm can be configured to use limited cores. Any idea how to do this configuration ? Which erlang option to use? Any examples etc.

Michael Klishin

unread,
May 3, 2017, 9:35:19 AM5/3/17
to rabbitm...@googlegroups.com

On Wed, May 3, 2017 at 2:36 PM, Chao Xu <sukka...@gmail.com> wrote:
Beam.smp is causing very high CPU usage and impacting other applications. From other threads I heard that Erlange vm can be configured to use limited cores. Any idea how to do this configuration ? Which erlang option to use? Any examples etc.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send an email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
MK

Staff Software Engineer, Pivotal/RabbitMQ

Luke Bakken

unread,
May 3, 2017, 1:15:14 PM5/3/17
to rabbitmq-users
Hi Chao,

Be sure to do some internet searching to see if you are experiencing high CPU due to how you are using RabbitMQ. For instance, a quick Google search using the terms "rabbitmq cpu usage" turned up these issues using Celery with RabbitMQ:


Chao Xu

unread,
May 3, 2017, 1:21:18 PM5/3/17
to rabbitmq-users
Thanks, Michael. which option should be set to limit the number of cores?


On Wednesday, May 3, 2017 at 8:35:19 AM UTC-5, Michael Klishin wrote:
On Wed, May 3, 2017 at 2:36 PM, Chao Xu <sukka...@gmail.com> wrote:
Beam.smp is causing very high CPU usage and impacting other applications. From other threads I heard that Erlange vm can be configured to use limited cores. Any idea how to do this configuration ? Which erlang option to use? Any examples etc.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send an email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Chao Xu

unread,
May 3, 2017, 1:26:46 PM5/3/17
to rabbitmq-users
i am not using celery. Just rabbitmq on redhat7

Luke Bakken

unread,
May 3, 2017, 1:39:04 PM5/3/17
to rabbitmq-users
Chao -

OK. My point, however, is that limiting CPU cores may only solve the symptom you are seeing (high CPU usage), rather than the cause.

Michael Klishin

unread,
May 3, 2017, 1:49:14 PM5/3/17
to rabbitm...@googlegroups.com
Add the following to your rabbitmq-env.conf:

SERVER_ADDITIONAL_ERL_ARGS="+S 2:2"

if you want the node to only use 2 cores. Change the 2s to 4s if you want 4 cores, and so on:

SERVER_ADDITIONAL_ERL_ARGS="+S 4:4"

Then in the logs you will see a line similar to this:

Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

and the smp part tells you how many schedulers are used by the Erlang VM (it's an oversimplification
but assume that one scheduler uses 1 core).

As Luke points out, it may be a better idea to investigate what causes the load
and why RabbitMQ is co-hosted with out applications that may be competing
for CPU resources with it :)

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
 To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Chao Xu

unread,
May 3, 2017, 3:16:07 PM5/3/17
to rabbitmq-users
thanks Michael. 


On Wednesday, May 3, 2017 at 12:49:14 PM UTC-5, Michael Klishin wrote:
Add the following to your rabbitmq-env.conf:

SERVER_ADDITIONAL_ERL_ARGS="+S 2:2"

if you want the node to only use 2 cores. Change the 2s to 4s if you want 4 cores, and so on:

SERVER_ADDITIONAL_ERL_ARGS="+S 4:4"

Then in the logs you will see a line similar to this:

Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

and the smp part tells you how many schedulers are used by the Erlang VM (it's an oversimplification
but assume that one scheduler uses 1 core).

As Luke points out, it may be a better idea to investigate what causes the load
and why RabbitMQ is co-hosted with out applications that may be competing
for CPU resources with it :)
On Wed, May 3, 2017 at 7:15 PM, Luke Bakken <lu...@bakken.io> wrote:
Hi Chao,

Be sure to do some internet searching to see if you are experiencing high CPU due to how you are using RabbitMQ. For instance, a quick Google search using the terms "rabbitmq cpu usage" turned up these issues using Celery with RabbitMQ:


http://stackoverflow.com/questions/6362829/rabbitmq-on-ec2-consuming-tons-of-cpu

On Wednesday, May 3, 2017 at 5:36:08 AM UTC-7, Chao Xu wrote:
Beam.smp is causing very high CPU usage and impacting other applications. From other threads I heard that Erlange vm can be configured to use limited cores. Any idea how to do this configuration ? Which erlang option to use? Any examples etc.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Chao Xu

unread,
May 3, 2017, 5:58:29 PM5/3/17
to rabbitmq-users
fyi, after the configuration, logs do not show smp:2:2 . It shows "Halting ERLang Vm "


On Wednesday, May 3, 2017 at 12:49:14 PM UTC-5, Michael Klishin wrote:
Add the following to your rabbitmq-env.conf:

SERVER_ADDITIONAL_ERL_ARGS="+S 2:2"

if you want the node to only use 2 cores. Change the 2s to 4s if you want 4 cores, and so on:

SERVER_ADDITIONAL_ERL_ARGS="+S 4:4"

Then in the logs you will see a line similar to this:

Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

and the smp part tells you how many schedulers are used by the Erlang VM (it's an oversimplification
but assume that one scheduler uses 1 core).

As Luke points out, it may be a better idea to investigate what causes the load
and why RabbitMQ is co-hosted with out applications that may be competing
for CPU resources with it :)
On Wed, May 3, 2017 at 7:15 PM, Luke Bakken <lu...@bakken.io> wrote:
Hi Chao,

Be sure to do some internet searching to see if you are experiencing high CPU due to how you are using RabbitMQ. For instance, a quick Google search using the terms "rabbitmq cpu usage" turned up these issues using Celery with RabbitMQ:


http://stackoverflow.com/questions/6362829/rabbitmq-on-ec2-consuming-tons-of-cpu

On Wednesday, May 3, 2017 at 5:36:08 AM UTC-7, Chao Xu wrote:
Beam.smp is causing very high CPU usage and impacting other applications. From other threads I heard that Erlange vm can be configured to use limited cores. Any idea how to do this configuration ? Which erlang option to use? Any examples etc.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Michael Klishin

unread,
May 3, 2017, 7:17:22 PM5/3/17
to rabbitm...@googlegroups.com
You need to restart the node for environment variable values to have an effect.

"Halting Erlang VM" is the last thing RabbitMQ logs on shutdown.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Chao Xu

unread,
May 4, 2017, 12:01:34 PM5/4/17
to rabbitmq-users
Did the $systemctl restart rabbitmq-server
Logs did not show [smp:2:2] that i expected. 
if I do $rabbitmqctl status then i see that erlang is still using 4 cores. Any idea if i missed any step?

Michael Klishin

unread,
May 4, 2017, 12:07:25 PM5/4/17
to rabbitm...@googlegroups.com
Are you using a distribution that uses systemd?

systemd has its own way of configuring environment variables, system limits,
and just about everything else.

This guide has nothing to do with RabbitMQ but mentions how it works:

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages