List of metrics for Scorecard Reports

[Damon Edwards]

Below is the list of the metrics for the new version of the Scorecard
Reports...

Alex, can you fill in how Service Management Controls (currently
called "Control Average") and Process Coverage are currently
calculated?


*******************

__Service Info__

Name: Goals Met
How calculated: percentage of goals met
Displayed as: percentage
Good: higher

Name: Risk
How calculated: TBD
Displayed as:
Good: lower



__Audit Scores__

Name: Service Management Controls
How calculated:
Displayed as: 0-100 score
Good: higher

Name: Process Coverage
How calculated:
Displayed as: 0-100 score
Good: higher

Name: Process Quality
How calculated: average of the quality measurements across all
process for service (consistency, effectiveness, reliability, safety,
success)
Displayed as: 0-100 score
Good: higher


__Performance__

Name: Service Availability
How calculated: Percentage of time that service was available (from
availability receipts)
Displayed as: percentage
Good: higher

Name: Estimated Outage Cost
How calculated: total time service was unavailable times estimated
hourly cost (some way to calculate partial outages?)
Displayed as: currency
Good: lower

Name: Mean Time To Repair (MTTR)
How calculated: mean amount of time it takes to restore service
availability (from availability receipts)
Displayed as: hours
Good: lower

Name: Mean Time Between Failure
How calculated: Mean time between service events where service has
availability issues (from availability receipts)
Displayed as: hours
Good: lower

Name: Service Failures (Due to Service Management Activity)
How calculated: Number of service availability failures/impairments
caused by service management activity (from availability receipts)
Displayed as: integer
Good: lower

Name: Service Failures (Total)
How calculated: Number of service availability failures/impairments
overall (from availability receipts)
Displayed as: integer
Good: lower

_Service Management Activity_

Name: Service Management Activities
How calculated: Total number of service management activities (from
process receipts)
Displayed as: integer
Good: higher (lower?... depends.. not necessarily good or bad?)

Name: High Impact Activity
How calculated: percentage of service management activities that were
marked high impact (from process receipts)
Displayed as: percentage
Good: lower

Name: Ad Hoc Activity
How calculated: Service management activities for which a process
receipt was submitted but it didn't follow a predefined process (from
process receipts)
Displayed as: integer
Good: lower

Name: Process deviations
How calculated: Service management activities that started with a
defined process but deviated from that process during execution (from
process receipts)
Displayed as: integer
Good: lower

Name: Service Management Success Rate
How calculated: Percentage of all service management activities that
completed with a successful status(from process receipts)
Displayed as: percentage
Good: lower

Name: Service Management Activities That Impacted Availability
How calculated: Service management activities that had an impact on
service availability (from process receipts only or from both process
and availability receipts?)
Displayed as: integer
Good: lower

Name: Service Management Activities That Where Unplanned
How calculated: Service management activities that didn't go through
proper planning and scheduling process (from process receipts)
Displayed as: integer
Good: lower

Name: Service Management Activities That Didn't Go Through Approval
Process
How calculated: Service management activities that didn't go through
the proper review and approval process
Displayed as: integer
Good: lower

[Alex-SF]

Control Average currently has a very rudimentary formula (and hardly
worth mentioning :))
Process Coverage Average is based on how many processes you've
implemented divided by the total possible.

[Greg Schueler]

OK, i've started up a google docs spreadsheet to keep track of this:

http://spreadsheets.google.com/ccc?key=psdknBBawNcxnlNiGA1Y-Sg

Greg

[Greg Schueler]

would it make sense to just treat those items which are listed as "0-100 score" as percentage (0%-100%)?

I already did change the "Process Coverage" metric to be a percentage, based on Alex's comment that it is "based on how many processes you've implemented divided by the total possible"

Greg

[Damon Edwards]

That makes sense. Percentage is clearer in what it is trying to show.

[Greg Schueler]

Ok, going with that.

Greg

[Greg Schueler]

Noticed one thing:  MTBF is listed as "lower is better", however I think it should be higher is better, right? I've made that change in the spread sheet

Greg

[Damon Edwards]

Yeah that was a mistake... Higher is better.

[Greg Schueler]

these two metrics seem to be the same:

"Service Failures (Due to Service Management Activity)"

and "Service Management Activities That Impacted Availability"

is there a preference on which one to keep?  if not I'll keep the first one

Greg

[Damon Edwards]

I'd go with the second one. When looking to improve service management you want to know how many of your sm activities were detrimental to your service. 

In any case, "failure" is difficult to use since it implies that the site was down hard. Brownouts where their is a reduced quality of service are much more common. 

[Greg Schueler]

ok, sounds good

Greg

[Greg Schueler]

another change: Service Management Success Rate, changed to "Good: higher".

Greg