[erlang-questions] 1PB of arg_reg - wait what?

[Simon MacMullen]

We've just had a report of a Rabbit broker dying trying to allocate
approximately 1PB of memory:

http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-May/019896.html

Slogan: std_alloc: Cannot allocate 1124304355803072 bytes of memory (of
type "arg_reg").

Which is unfortunate enough in itself - but arg_reg? My understanding of
beam internals is not great, but that looks to be something like a stack
frame. A 1PB stack frame?

As far as I can see "arg_reg" memory is only allocated in one place,
around line 3100 of beam_emu.c:

https://github.com/erlang/otp/blob/maint/erts/emulator/beam/beam_emu.c#L3157

...where size is set to c_p->arity * sizeof(c_p->arg_reg[0]).

process.arg_reg is an Eterm*, so sizeof(Eterm) *cannot* be huge, so it
must be the arity which is ~10^15. We have some excessive arities in
RabbitMQ, but not quite to that extent.

Does anyone have any idea how this could arise, or can it just be
chalked up to cosmic rays?

Cheers, Simon

--
Simon MacMullen
RabbitMQ, VMware
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Max Lapshin]

99% that some error in process with many terms leads to error_logger
eating all possible CPU and memory.

[Emile Joubert]

On 08/05/12 20:31, Max Lapshin wrote:
> 99% that some error in process with many terms leads to error_logger
> eating all possible CPU and memory.

A RabbitMQ customer claims to have reproduced this error even after
disabling logging. (The thread from the rabbitmq-discuss mailing list
starts here:
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2013-September/029829.html)

The error is:

=erl_crash_dump:0.2
Mon Sep 9 12:01:31 2013
Slogan: std_alloc: Cannot allocate 1121150602465920 bytes of memory (of
type "arg_reg").
System version: Erlang R16B01 (erts-5.10.2) [source] [64-bit]
[async-threads:30] [hipe] [kernel-poll:true]
Compiled: Sun Jul 7 05:35:15 2013
Taints:
Atoms: 27430


Is there anything that can be done to avoid errors like this?
Is there any way of determining what the original error was that
triggered the crash?




-Emile

[Sverker Eriksson]

On 05/08/2012 04:30 PM, Simon MacMullen wrote:
> We've just had a report of a Rabbit broker dying trying to allocate
> approximately 1PB of memory:
>
> http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-May/019896.html
>
> Slogan: std_alloc: Cannot allocate 1124304355803072 bytes of memory
> (of type "arg_reg").
>
> Which is unfortunate enough in itself - but arg_reg? My understanding
> of beam internals is not great, but that looks to be something like a
> stack frame. A 1PB stack frame?
>
> As far as I can see "arg_reg" memory is only allocated in one place,
> around line 3100 of beam_emu.c:
>
> https://github.com/erlang/otp/blob/maint/erts/emulator/beam/beam_emu.c#L3157
>
>
> ...where size is set to c_p->arity * sizeof(c_p->arg_reg[0]).
>
> process.arg_reg is an Eterm*, so sizeof(Eterm) *cannot* be huge, so it
> must be the arity which is ~10^15. We have some excessive arities in
> RabbitMQ, but not quite to that extent.
>
> Does anyone have any idea how this could arise, or can it just be
> chalked up to cosmic rays?
>

c_p->arity in this case is supposed to be the number of arguments to
some function or fun.
The allocation is made to save the function arguments before the process
is scheduled out.

I seems the c_p->arity has got an insanely large value for some reason.

A core dump would be more helpful to track down this bug. Command line
option +d can be used to skip the erl_crash_dump and get a more usable
core dump.

/Sverker
Erlang/OTP Ericsson

[Anthony Ramine]

Out of curiosity, why is arity an Uint and not a byte? Erlang functions can't have more than 255 arguments, right?

Le 9 sept. 2013 à 19:12, Sverker Eriksson a écrit :

> I seems the c_p->arity has got an insanely large value for some reason.