[erlang-questions] How to introspect atom table?

[Max Lapshin]

Erlang system is crashing with atom limit.
Is it possible to look inside atom table, maybe I'll see something like: 'user_15', 'user_157' and will quickly find source of problems.

[Gleb Peregud]

AFAIK crash dump contains atom list.

Also erlang:memory(atom) or erlang:memory(atom_used) can be useful

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Max Lapshin]

Atoms: 1048576
=memory
total: 507885448
processes: 336611825
processes_used: 336601278
system: 171273623
atom: 52749265
atom_used: 52707589
binary: 6311816
code: 3898230
ets: 2209656
=hash_table:atom_tab
size: 411527
used: 322545
objs: 1048577
depth: 21


This looks like a problem, yes?

[Gleb Peregud]

Yes. 1048576 is 2^20 which seems to be a size of atom table (atom.h:30):

#define ATOM_LIMIT (1024*1024)

[Max Lapshin]

Problem was in peb: http://code.google.com/p/mypeb/source/browse/trunk/peb.c#359

guys are using it and it creates new nodename each time it connects. Atom table quickly gets exhausted.

[Zabrane Mickael]

Did you try increase this limit:

$ erl ... +t 4194304

Regards,

Zabrane

[Max Lapshin]

It is a bad idea to increase atom limit, because it will just prolong the suffering.
1 million of atoms is enough for a simple program.

[Zabrane Mickael]

On Nov 20, 2012, at 1:11 PM, Max Lapshin wrote:

> It is a bad idea to increase atom limit, because it will just prolong the suffering.

So fix the code and try to use list_to_existing_atom/1instead of list_to_atom/1 (see http://erlang.org/doc/man/erlang.html).
Better, redesign the code.

> 1 million of atoms is enough for a simple program.

In our case it wasn't sufficient.

Regards,
Zabrane

[Max Lapshin]

I've written, that problem wasn't in converting string to atoms, it was in external PHP code that created unique node names on each request.

[Loïc Hoguin]

We had the issue with peb too and just stopped using it entirely.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Max Lapshin]

I've told to switch in peb from connect to pconnect. Usually it helps in PHP

On Tue, Nov 20, 2012 at 4:46 PM, Loïc Hoguin <es...@ninenines.eu> wrote:

On 11/20/2012 01:04 PM, Max Lapshin wrote:

Problem was in peb:
http://code.google.com/p/mypeb/source/browse/trunk/peb.c#359

guys are using it and it creates new nodename each time it connects.
Atom table quickly gets exhausted.

We had the issue with peb too and just stopped using it entirely.

--
Loďc Hoguin

[Loïc Hoguin]

Yes but that's not going to stop peb from making atoms, it'll just
create them slower. PHP processes are restarted every few thousand
requests, which forces a reconnect, and the connection might fail, which
reconnects it too.

It'll just be a slower time bomb.

On 11/20/2012 02:00 PM, Max Lapshin wrote:
> I've told to switch in peb from connect to pconnect. Usually it helps in PHP
>
>
> On Tue, Nov 20, 2012 at 4:46 PM, Loïc Hoguin <es...@ninenines.eu
> <mailto:es...@ninenines.eu>> wrote:
>
> On 11/20/2012 01:04 PM, Max Lapshin wrote:
>
> Problem was in peb:

> http://code.google.com/p/__mypeb/source/browse/trunk/peb.__c#359

> <http://code.google.com/p/mypeb/source/browse/trunk/peb.c#359>
>
> guys are using it and it creates new nodename each time it connects.
> Atom table quickly gets exhausted.
>
>
> We had the issue with peb too and just stopped using it entirely.
>
> --
> Loďc Hoguin
> Erlang Cowboy
> Nine Nines
> http://ninenines.eu
>
>

--
Loïc Hoguin

Erlang Cowboy
Nine Nines
http://ninenines.eu

[Max Lapshin]

But there are only 32 000 of possible pids in unix, so it will limit amount of different names.

[Jachym Holecek]

# Max Lapshin 2012-11-20:

> But there are only 32 000 of possible pids in unix, so it will limit amount
> of different names.

I'm curious -- do you have a reference for this limit? POSIX doesn't seem to
define pid_t range.

BR,
-- Jachym

[Max Lapshin]

On Tue, Nov 20, 2012 at 5:50 PM, Jachym Holecek <fr...@circlewave.net> wrote:

# Max Lapshin 2012-11-20:

> But there are only 32 000 of possible pids in unix, so it will limit amount
> of different names.

I'm curious -- do you have a reference for this limit? POSIX doesn't seem to
define pid_t range.

No, of course =)
But I haven't ever seen 32 bit pid, so it is ok for this case.

[Daniel Goertzen]

On linux a pid_t is 32 bits, but the limit is configurable.  My Ubuntu 64 box says...

$ cat /proc/sys/kernel/pid_max 

32768

Dan.

[Max Lapshin]

ook. It would be a big problem with my hack-patch if this limit would be raised =))

[Richard O'Keefe]

An actual transcript from the machine I am typing this on.

m% ps -a
PID TTY TIME CMD
25493 ttys000 0:00.10 login -pf ok
25494 ttys000 0:00.64 -bash
39476 ttys000 1:52.02 find /home/cshome/o/ok -name pperl.c -print
70511 ttys000 0:00.04 view UNPACK.S
25500 ttys001 0:00.01 login -pfq ok /usr/bin/ssh frege
25501 ttys001 0:03.10 -ssh frege
40195 ttys002 0:00.10 login -pf ok
40196 ttys002 0:00.88 -bash
83669 ttys002 0:00.32 /home/cshome/o/ok/local/lib/erlang/erts-5.6.3/bin/beam
55602 ttys003 0:00.03 login -pf ok
55603 ttys003 0:25.55 -bash
86257 ttys003 0:00.00 ps -a
59007 ttys004 0:00.02 login -pfq ok /usr/bin/ssh ok@frege
59008 ttys004 0:00.90 -ssh ok@frege

Almost all of those Unix pids exceed 32,000.
Two of them even exceed 16 unsigned bits.

This is a 4 GiB machine with a single user.

[Max Lapshin]

Ok, there are two ways:

1) tell php programmer to change connect to pconnect in his buggy and horrible piece of php code and take a look at what will happen

2) tell php programmer truth about his piece of software and advice him to completely rewrite this subsystem

First is easier a bit.