Mix OTP behaviour graph idea

[Chris Keele]

I've had an idea for a little bit of tooling for a while, but I can't think of a good way to implement it outside of core.

When I heard about mix xref graph it reoccured to me. I'm wondering what other people think about bridging that concept with :observer's process link graph, to generate a non-runtime view of what a project's OTP behaviour structure looks like.

The end product I envision is an ERD PDF artifact in project root that graphs what different OTP behaviours compiled modules have implemented, and how they relate to each other. An example might plot the project's main OTP Application module as a root node, that branches out to show the names of Supervisors it starts and their Workers, plotted with different shapes if they're GenEvent handlers, GenServers, or Agents; and perhaps connected with different line types depending on their restart type. Modules that start at Application boot like loggers and Ecto repos would optionally show up alongside the rest. Sort of a map of project entities and their responsibilities.

The CLI API I envision is similar to rails-ERD's rake task, namely flags and options to exclude modules of different names/namespaces, behaviour types, and origins (internal versus external).

Note I'm not talking about a compile-time graph of all processes and links, so much as a high-level overview of project structure. Just being able to see a project's core OTP behaviours at a glance diagrammed and related to each other would help onboarding, whiteboarding, and project design.

Of all the implementations I've brainstormed, the only pleasant one that occurs to me would be core taking possession of some module attribute names for this purpose, and have it such that our Supervisor, Behaviour, GenServer, etc injects proper attributes into using modules. Similarly, the Supervisor.Spec macros could inject the relational information required into the module.

If the structure of the module attributes are designed well, this would enable an extensible API for other projects with their own Behaviour types to insert new entity types into the graph, and let end users identify and annotate project-specific custom behaviours as well.

I don't see a tenable way to have this exist outside of core as a library without asking users to exclusively use wrapper GraphableOTP.Supervisors and so on for everything, so I'm fielding it here rather than pursuing a proof-of-concept library first. :) Thoughts?

[Paul Schoenfelder]

The BEAM contains metadata such as what behaviors a module implements and that would be enough to determine what entities they represent. The tough part is being able to trace what the links between them are. As far as I know, there's no way to determine (from the BEAM attributes) what processes a given supervisor starts, but perhaps there's another way.

Paul 

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/f2692324-42ef-423c-9f74-e39380cc1179%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Keele]

It'd be pretty cool if it could just run on compiled BEAM source code! That'd make it erlang interoperable too.

The relations between them is the challenge. Do you know if module attributes are introspectable in BEAM source? If so, the task could run on those too, and erlang/elixir users could manually annotate their modules, as well as/instead of Supervisor.Spec injecting them for you. Then it'd be super extensible for other usecases, too.

[José Valim]

I believe this information best comes from runtime. So you should rather start the application and then traverse the supervision tree, collecting information like children PIDs, names, behaviours and so on. That should be interporable too and it is pretty close to how observer works.

I think relying on the module attributes is quite limited because you cannot rely build the hierarchy from it. Generating a graph could be useful but I don't see how it would be superior in anyway to observer.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

On Fri, Jun 24, 2016 at 9:18 PM, Chris Keele <d...@chriskeele.com> wrote:

It'd be pretty cool if it could just run on compiled BEAM source code! That'd make it erlang interoperable too.

The relations between them is the challenge. Do you know if module attributes are introspectable in BEAM source? If so, the task could run on those too, and erlang/elixir users could manually annotate their modules, as well as/instead of Supervisor.Spec injecting them for you. Then it'd be super extensible for other usecases, too.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/66dfb6e3-0d27-4d2f-ae0d-0b1e47bf2ef1%40googlegroups.com.

[Chris Keele]

I believe this information best comes from runtime... Generating a graph could be useful but I don't see how it would be superior in anyway to observer.

Yeah, definitely don't want to reinvent observer, just have a mind map graph of the key runtime elements as a canonical reference available outside of runtime. Maybe I'll explore starting, interacting with, snapshotting, and graphing observer from inside a mix task, instead. That'd be nice and outside of core, although the exact opposite of the run-on-compiled-BEAM-code approach.