Proposal: Expose multiple metrics in reports

[Jeremy Dunck]

django-lean uses a single metric for each report. I can see the value
in this-- a single number is easier to make an improvement decision
from.

But it assumes that you can get buy-in from your organization
regarding the function that creates the single metric.

Do you have advice in how to arrive at this metric? If not, what do
you think of adding functionality to keep metric parameters separated
for reporting purposes? I guess this would lead either to separate
confidences as well.

[Erik Wright]

Interesting questions.

I believe there is value in reporting on more than one metric, but
that there should be one primary metric that determines "traction" and
that measures the progress of your customer development process. The
other metrics are there to provide additional insight on _why_ the
experiment succeeded or failed.

Occassionally, an experiment that improves overall traction might have
a negative impact on some specific metric. In that case, you might
continue experimenting in the hopes of achieving the benefits without
the side-effects.

I definitely think the system should support multiple metrics, with
every metric calculated nightly for every active experiment. Perhaps
the system could automatically discover them by looking for a lean.py
file in each app?

If you are planning to implement this let me know and I'll provide
more details on how this could be integrated best.

Cheers,

Erik

[Jeremy Dunck]

On Wed, Oct 14, 2009 at 12:14 AM, Erik Wright <er...@erikwright.com> wrote:
...

> If you are planning to implement this let me know and I'll provide more
> details on how this could be integrated best.

Let me use django-lean for real for a bit and see if I really have
this problem. But thanks for the feedback. :-)