[erlang-questions] Request for opinions on using process dictionary for meta information

[Jay Doane]

Our team is currently using the process dictionary to store meta information for long running processes, and would like to hear from folks familiar with beam internals whether this is a recommended practice.

Essentially we add meta information about the current request in a process dictionary of every process related to handling of this request. This information could be:

- current file

- user name of initiator of the request

- http query parameters

- type of a process (indexer | compactor | http_handler)

For example, when we detect performance problems, we run a function that returns the Pids of (say) top memory using processes. We use the meta info stored in process dictionaries to determine which documents are using excessive memory so we can take corrective action.

In practice, we are unable to use normal OTP messaging because in many cases, the high memory using processes have also grown huge message queues, and so would be unresponsive to any additional messages.

Given our existing constraints, is this a reasonable use of the process dictionary?

Thanks,

Jay

[Dmytro Lytovchenko]

If you store some data that doesn't change much later — I see no problem here, perfectly fine example of PD use.

But on the other hand, if you store mutable data in PD — this is considered an antipattern and it makes your code have side effects which are hard to track. Also when your process dies, all data in PD will be lost, which is OK if data is set once, but not OK if the data was important.

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Max Lapshin]

Yes, this is a good way.

You can inspect process in such non-blocking way, but use this information as a hint, not as a proper and reliable data.

[Mikael Pettersson]

Yes, I think this is perfectly reasonable.

[Gianfranco Alongi]

Hi!

It kind of sounds similar to the idea behind gproc, which is a widely used library,
so as everyone has said already - this is perfectly fine.

https://github.com/uwiger/gproc

Use case: System inspection

Gproc was designed to work as a central index for "process metadata", i.e. properties that describe the role and characteristics of each process. Having a single registry that is flexible enough to hold important types of property makes it easier to (a) find processes of a certain type, and (b) query and browse key data in a running system.

[Roger Lipscombe]

On 18 April 2017 at 10:29, Gianfranco Alongi <gianfran...@gmail.com> wrote:

Hi!

It kind of sounds similar to the idea behind gproc, which is a widely used library,
so as everyone has said already - this is perfectly fine.

https://github.com/uwiger/gproc

Doesn't gproc store the process metadata in an ETS table? That's a valid approach, too, but you need a monitor process to clean up if the process dies.

As another pro-example of storing information in the process dictionary; this is what lager does with metadata -- see lager:md.

Both approaches are valid: we use all of the following:

1. Process dictionary (usually for lager, but sometimes separately).

2. Separate process registry using one or more ETS tables (we don't use gproc; we wanted slightly different semantics).

3. Process responds to gen_server:call(Pid, info)

They're useful for different information, for different scenarios.

I'd be reluctant to use the process dictionary for anything other than process-internal-use or debugging information, though.