I hope to talk to one of Sun's DTrace guru's this week. Using  
Erlang's trace/3 seems to be a key issue. I'll add a bit extra text  
to help in case someone unfamiliar with DTrace and Erlang is reading  
this mail.

Erlang already has a powerful trace mechanism, erlang:trace/3  
available directly to the programmer from the Erlang programming  
language. Erlang's trace/3 built in function (BIF) can trigger  
tracing of both Erlang application and VM behaviour, including  
module:function calls, Erlang process tracing as well as internal VM  
behaviour like garbage collection.

The fundamental question is:
Is there a practical way to switch on Erlang trace/3 functionality  
when a DTrace script is executed, and later turn off the Erlang trace/
3 functionality when the DTrace script exits?

Some of the hopes for this route are:
- Erlang programmers would quickly see how to use Erlang DTrace,
- any new trace/3 functionality would be quickly made available to  
Erlang DTrace,
- some 'thorny issues' are already solved in Erlang trace/3 so this  
path reuses those solutions

There are several types of Erlang DTrace probes.
It is not *essential* for Erlang DTrace providers to interact with  
Erlang's trace/3 functionality, but it may be the best path for some  
types of probe.

One type of probe will observe the underlying VM, which is largely  
invisible to the Erlang code. These probes appear to have concrete  
implementation in the C source of the VM, and so is amenable to  
instrumentation with DTrace static probes (Chapter 34 of 817-6223).  
Even here though, the trace/3 BIF may be useful, for example it  
supports tracing of Erlang process garbage collection.

The other type of probe is for the Erlang source code. This is the  
area less visible with DTrace static probes because Erlang code is  
interpreted (ignoring HiPE). Ideally probes would reveal Erlang  
source-related events, for example module:function entry and exit,  
message send and receive, process spawn, exception events, etc. These  
probes are probably the hardest to implement from scratch. They would  
likely gain the most benefit from using the Erlang trace/3 BIF.

Erlang's trace/3 needs parameters to define the the scope of tracing.
So, the fundamental questions are,
1. how do trace/3 parameters get moved from a D script into calls of  
trace/3?

Supplementary question: trace/3 documentation says "Only one process  
can trace a particular process. For this reason, attempts to trace an  
already traced process will fail". If a second D script is started,  
and it tries to trace processes which are already being traced,  
should the second script fail (if so how), or should Erlang DTrace  
support concurrent tracing (which is in the spirit of DTrace). A  
related question is  what happens if Erlang program is already  
tracing, must the D script fail?

2. how does trace/3 get reset (i.e. switched off) when a D script ends?

Supplementary question: if Erlang DTrace supports concurrent tracing,  
and two D scripts are running concurrently, and are using overlapping  
probes, how do the common probes survive (stay on) when one of the D  
scripts exits?

Even with these questions solved effectively, there are still  
questions about how to use trace/3. Specifically how to intercept  
trace/3 output and get the resulting information out to DTrace, and  
how to represent those Erlang values to DTrace. These questions feel  
less fundamental. For example, trace/3 output is text, so Erlang  
DTrace could convert Erlang values to a pretty-printed text form.  
This may be less efficient than ideal, but would be no worse than any  
other external trace analysis tool.

GB-)