Code and Build Conventions

[G Bulmer]

Tim

It seemed worthwhile to start this thread.

We need some code and development conventions, and you are likely to
get to code first.

I assume we will wrap code changes to the base Erlang distribution is
C '#if define(...) ... #endif'.
We should be able to build without our DTrace support, and get an
equivalent/identical build result to the erlang.org development release.

You choose the conventions, I thought ERLANG_DTRACE or ERL_DTRACE as
a macro name.

It'd be good to have a consistent flag to make too, so that we can
easily make a normal or 'DTrace-enhanced' build.

I assume you'll figure out an approach to the directory structure. I
don't know what your views are.

Please send thoughts to the group for the benefit of 'future
generations' ;-)

Garry

[Tim]

> We need some code and development conventions, and you are likely to
> get to code first.

Agreed on the first point, not sure about the second.

> I assume we will wrap code changes to the base Erlang distribution is
> C '#if define(...) ... #endif'.
> We should be able to build without our DTrace support, and get an
> equivalent/identical build result to the erlang.org development release.
>
> You choose the conventions, I thought ERLANG_DTRACE or ERL_DTRACE as
> a macro name.

Once again, I scavenged through the Ruby implementation to see how
they do things. Both Ruby and OTP use make + autotools as build system
and it seems as easy as adding:

+AC_ARG_ENABLE(dtrace,
+ [ --enable-dtrace enable DTrace support.],
+ [enable_dtrace=$enableval])
+if test "$enable_dtrace" == "yes" -a "$ac_cv_header_sys_sdt_h" ==
"yes"; then
+ AC_DEFINE(ENABLE_DTRACE)
+ DTRACE_OBJS="dtrace.o"
+else
+ DTRACE_OBJS=""
+fi
+AC_SUBST(DTRACE_OBJS)


To config.in and regenerating the `configure` script. The way I
understand it, this leads to a ENABLE_DTRACE macro to be available in
the generated `config.h`. I'll have a look around to see if that's the
"standard" naming convention for the macro name.

> It'd be good to have a consistent flag to make too, so that we can
> easily make a normal or 'DTrace-enhanced' build.

The above should allow you to make the otp dist with:

./configure --enable-dtrace
make
make install

> I assume you'll figure out an approach to the directory structure. I
> don't know what your views are.

I don't think we'll require any new files, just some patches to the
existing ones, so the only problem with directory structure is
figuring out the existing structure of the otp distribution.

[Tim]

I just checked postgrsql's config.in

#
# DTrace
#
PGAC_ARG_BOOL(enable, dtrace, no,
[ --enable-dtrace build with DTrace support],
[AC_DEFINE([ENABLE_DTRACE], 1,
[Define to 1 to enable DTrace support. (--enable-dtrace)])
AC_CHECK_PROGS(DTRACE, dtrace)
if test -z "$DTRACE"; then
AC_MSG_ERROR([dtrace not found])
fi
AC_SUBST(DTRACEFLAGS)])
AC_SUBST(enable_dtrace)



I guess ENABLE_DTRACE is the 'consensus' macro name. We can always
change it later.

[G Bulmer]

Tim

That's nice. Well done for planning so far ahead ;-)

>> It'd be good to have a consistent flag to make too, so that we can
>> easily make a normal or 'DTrace-enhanced' build.
>
> The above should allow you to make the otp dist with:
>
> ./configure --enable-dtrace
> make
> make install

Cool. If that's all there is to it, then it's done! You're having a
very productive Friday, man.

>> I assume you'll figure out an approach to the directory structure. I
>> don't know what your views are.
>
> I don't think we'll require any new files, just some patches to the
> existing ones, so the only problem with directory structure is
> figuring out the existing structure of the otp distribution.

Really?
Based on an initial 'skim' of the documentation, I can imagine
'fixed' providers like garbage collection events, and scheduling
events might all fit into the existing source, but I had imagined
that there might be a bit of glue needed for other events. I'm not
certain, but, for example, there may be a need for code to do type
conversion to support getting at the Erlang programs functions and
arguments. I imagined that stuff would be nicer in its own directory.
Also, I imagined we'd have some test cases (e.g. Erlang programs,
data and D scripts), and they'd be in a separate directory.

Of course, we don't have to solve everything now.

Garry

[Tim]

> Really?
> Based on an initial 'skim' of the documentation, I can imagine
> 'fixed' providers like garbage collection events, and scheduling
> events might all fit into the existing source, but I had imagined
> that there might be a bit of glue needed for other events. I'm not
> certain, but, for example, there may be a need for code to do type
> conversion to support getting at the Erlang programs functions and
> arguments. I imagined that stuff would be nicer in its own directory.
> Also, I imagined we'd have some test cases (e.g. Erlang programs,
> data and D scripts), and they'd be in a separate directory.

It's just hubris on my part again, so let me rephrase: I don't think
we'll need *much* support code, but either way will obviously need
some place in the setup to park documentation, example D scripts,
etc.

[Tim]

> Based on an initial 'skim' of the documentation, I can imagine
> 'fixed' providers like garbage collection events, and scheduling
> events might all fit into the existing source

Just collecting random thoughts at the momemt:

the gc probe in trace/3 isn't as 'static' as one would think, in fact,
there's quiete a lot of information about the generational gc being
provided in the erlang trace messages:

>{trace, Pid, gc_start, Info}
> Sent when garbage collection is about to be started. Info is a list of two-element tuples, where the first element is a
>key, and the second is the value. You should not depend on the tuples have any defined order. Currently, the following
>keys are defined:

> heap_size
> The size of the used part of the heap.
> heap_block_size
> The size of the memory block used for storing the heap and the stack.
> old_heap_size
> The size of the used part of the old heap.
> old_heap_block_size
> The size of the memory block used for storing the old heap.
> stack_size
> The actual size of the stack.
> recent_size
> The size of the data that survived the previous garbage collection.
> mbuf_size
> The combined size of message buffers associated with the process.
>
> All sizes are in words.

I'm unconvinced that the information would be relevant very often in
practical usage, but it's fascinating to have from a nerdy "How Stuff
Works" perspective. I guess on the other hand, IF the problem you're
debugging is so complicated/convoluted that you actually need to probe
GC events, one would probably appreciate the extra information.

[G Bulmer]

Tim

DTracing Erang garbage collection seems to be a great use case to
focus details into clarity.

I think this type of stuff, as a return value, could be handled *if*
we can return a D associative array. I don't know how to do this.
Alternatively, we could have a *lot* of probe names, and use lots of
probe descriptions for an action.

Garry

[Tim]

Concerning "coding conventions" what I think what I'll do is only
modifying the original erlang code to the extent of:

#ifdef ENABLE_DTRACE
#include "dtrace/<probecode>
#endif

and moving all the dtrace specific code into a new `erts/emulator/beam/
dtrace` subdirectory just to keep things cleaner. I'm not really sure
though if that's the proper "C-way" of doing things, at least in the
erlang codebase they seem to like really long functions and files.
Either way, it shouldn't be very difficult to move code back and forth
in case of objections.

-tim