Calculating a Weighted Score?
Jarrod Hollingworth
only had one reply.
I'd be happy to give a free license to one of my software products (they are
not math related) for some help if that draws a crowd!:
- Domain Name Pro: http://www.mozzle.com
- Complete Time Tracking: http://www.complete-time-tracking.com
THE GOAL
I'm trying to devise a method for calcluating a "qualitative" weighted score
for items to indicate their degree of acceptance or rejection, whereby the
item is scored on a range of criteria, each with a different "importance".
The scoring of each items is done individually - the score is not used as a
comparative analysis of one item against another. Additionally, the scoring
of each criteria for an item is done independently (no comparison can be
done against other scores), though in some cases the criteria are somewhat
related. The set of criteria is not fixed (criteria can be added or removed
over time and new items are just scored on the current set of criteria,
existing items can be re-scored).
No, this is not a school/uni homework exercise! I am designing a software
application that needs a particular implementation of this idea.
So the inputs will be a list of criteria with weights and a score for each
criteria. The output will be a combined score in a fixed range.
I have been pointed to look at logistic regression for calculating the
weights to use for each criteria. I'm investigating that but I might be
happy using "gut feel" weights.
What I need help with is how to combine the weights and scores for each
criteria to:
a) Produce a final combined weighted score in a fixed range (say 0..1)
b) Give criteria with high weight and high score a lot more importance than
several low weight / low score criteria.
Upon doing some research I thought that it might be related to MCA or MCDA
(Multi Criteria Decision Analysis) as outlined here:
http://www.odpm.gov.uk/index.asp?id=1142259
My current idea is to assign each criteria a weight from 0..1, where 0 is
"not indicative" and 1 is "ultimately indicative". Each criteria would be
scored in the range -1 (very bad) to +1 (very good) with 0 being neutral.
EXAMPLE
Take the simple (contrived) example of accepting/rejecting a specific bottle
of wine (a single bottle selected from the cellar) for award of a gold
medal. Other bottles from the same source may have been previously judged.
The various criteria and "gut feel" weights might be:
1. Color of wine 0.3
2. Cloudiness/sediment in wine 0.4
3. Design of bottle 0.1
4. Design of label 0.2
5. Oxidization 1.0 (if wine is "off" then it is no good)
6. Previous judgement 0.9
7. Taste 0.7
8. Scent 0.9
RESULT
After each criteria is scored for the bottle the end result should be a
combined score based on the weight and score of all criteria and would range
from 0..1 (ie. not a simple open-ended sum of scores). Acceptance of the
item might be an overall score of 0.8 to 1.0, rejection 0.0 to 0.6, and
"inconclusive" 0.6 to 0.8. These acceptance ranges are just presentation
"fluff" and not intended to be part of the calculation itself.
A simple calculation for the weighted score might be calulated as the
average of all weighted criteria scores (criteria weight x criteria score).
QUESTIONS
1. Am I on the right track for implementing such a system?
2. Is there a better approach to simply averaging all weighted criteria
scores (to keep the final score in the range 0..1)?
3. How can I give more emphasis to high weight / high score criteria? Some
kind of exponential formula? (e.g. multiply square of weight and square of
score to get weighted criteria score perhaps?)
Thanks for your help.
--
Regards,
Jarrod Hollingworth