What scoring system would you use for this competition?

[Toby Pereira]

I know this is a bit off-topic, but it is connected to some of the issues discussed on here.

Each week, there is a competition and every competitor has a finishing position. But different numbers of people turn up each week, so we need a scoring system that takes this into account. For example, if the winner scored 100 points when 1000 people turn up, it might seem a bit odd if they scored 100 points when 3 people turn up. If we assume that the average quality of the competitors does not depend on the number of people (basically those that turn up are a random sample of those who could turn up), what would be a reasonable scoring system? Also, the score given can't be based on how well someone did other than their position. Winning by a lot scores the same as winning by a small amount. I have my own ideas, but I thought I'd gauge opinion before saying anything.

[esand...@gmail.com]

Why does the # of 'voters' change so drastically from week to week?

Can we assume that the # of voters from week to week is also a random sample?

[Clay Shentrup]

http://ScoreVoting.net/BetterQuorum.html

[Toby Pereira]

On Saturday, 6 February 2016 19:22:58 UTC, esand...@gmail.com wrote:

Why does the # of 'voters' change so drastically from week to week?

Can we assume that the # of voters from week to week is also a random sample?

Well, the number doesn't necessarily change that much from week to week - it was just to illustrate the point. Anyway, I would see the competitors as more analogous to candidates. The weekly competitions are the "voters", as they rank these "candidates".

But anyway, I've encountered a few competitions a bit like this that have an overall standings table based on how people have done over several competitions - whether its a weekly run, or an online competition where entry is fairly open. There are two basic ways I've seen it dealt with:

One is that the winner gets a fixed score (often 100), and then each place gets one point less than the position above. Second place would get 99 points down to 100th place getting 1 point, and then in some systems 101th and below get 0, or in others everyone from 100th down gets 1 point. The main disadvantage of this seems to be that people are over-rewarded for competing when not many other people do. Coming last of three people still gets you 98 points, whereas 10th of 100 gets you just 91.

The other is that last place gets 1 point, and each place higher up gets one more point. So the winner of 10 people gets 10 points, and the winner of 100 people gets 100 points. But I think this system over-rewards people for competing when lots of other people do. You get 10 points for winning out of 10, or for coming 91st out of 100.

I think a more sensible system would be to fit the competitors equidistantly between a theoretical minimum and maximum score - say 0 and 100. If there is just one competitor, they get 50 points. Two would get 67 and 33 (to the nearest integer). So the general formula for someone's points would be points = 100 - (position / (competitors + 1)*100). For linear scoring, I think this is the fairest way, and also the most sensible way of calculating percentiles (there seems to be no standard method).

For a non-linear system you could have something like points = (competitors + 1) / position, which rewards competitors more per position the higher up they finish.

[Toby Pereira]

On Saturday, 6 February 2016 20:16:19 UTC, Clay Shentrup wrote:

http://ScoreVoting.net/BetterQuorum.html

This could be a fair way of dealing with competitors who don't compete every week without putting them out of the running. Some other systems only count a competitor's best scores. For example, if there are 10 competitions that make up a championship, then it might be that only a competitor's best 7 count. The quorum rule could be quite interesting though.

Also, using a non-linear scoring system (like the one I mentioned in the post above) would mean that competitors could still make up enough points even if the miss a few competitions.

There is also using a Condorcet system, which would mean that someone could establish their position by turning up to just over half of the events. That would be a bit messy though, and cycles could make it quite ugly.

[esand...@gmail.com]

I prefer something more like the IMDb system:

http://stackoverflow.com/questions/2134504/what-is-the-best-algorithm-to-calculate-the-most-scored-item

[Clay Shentrup]

On Tuesday, February 9, 2016 at 2:47:29 PM UTC-8, esand...@gmail.com wrote:

I prefer something more like the IMDb system:

http://stackoverflow.com/questions/2134504/what-is-the-best-algorithm-to-calculate-the-most-scored-item

 If you look at the link "Mathematical underpinnings of this", you get to this:

http://scorevoting.net/PuzzDLaplace.html

Which mentions the IMDB system.

[Ted Stern]

Consider also "Lower bound of Wilson score confidence interval for a Bernoulli parameter", which is used by Reddit and other social media:

http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

--
You received this message because you are subscribed to the Google Groups "The Center for Election Science" group.
To unsubscribe from this group and stop receiving emails from it, send an email to electionscien...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Clay Shentrup]

On Tuesday, February 9, 2016 at 4:44:26 PM UTC-8, Dodecatheon Meadia wrote:

Consider also "Lower bound of Wilson score confidence interval for a Bernoulli parameter", which is used by Reddit and other social media:
http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

See Warren's post here.

https://groups.google.com/d/msg/electionscience/mzxHwRY--YA/HbqOpe40JtEJ

[esand...@gmail.com]

I prefer my 'no opinion' scoring system because it doesn't require any arbitrary quorums: https://groups.google.com/forum/#!searchin/electionscience/eric$20sanders/electionscience/SpLDc1Z0hzE/3FAIKr6OfVUJ (just updated post)

However, my system doesn't work for the example Toby gives. It's only meant for a 'single-round' election, where all voters have an opportunity to score all candidates.

[Toby Pereira]

Another question related to this is how to score competitors for an overall championship in situations where you do want to take their individual competition scores into account but where the individual scores can very wildly from event to event.

For example, in one competition, four people's scores might be:

1. 9

2. 6

3. 3

4. 0

But in another they might be:

1. 1000

2. 984

3. 980

4. 972

Note that it wouldn't make much sense to simply multiply the second set of scores by 9/1000 because you'd end up with a very small range rendering the second set of scores irrelevant. Another way of doing it would be to normalise the scores by setting the top score to, say, 100 and the bottom to 0 in all cases and have the rest spaced in the same proportions. But then in this case:

1. 100

2. 99

3. 98

4. 0

The scores would be unchanged, but ideally we'd have a way of mitigating the effect of the outlier.

As in the other example, I have my own idea of how to deal with this, but it would be interesting to see if anyone else has any clever solutions.

[Toby Pereira]

To finish this off - The way I'd want to do this would be to have a 0 to 100 scale like percentiles, but where the mean rather than the median is the 50th. I call them permeantiles. I'm surprised I've never encountered such a concept, although the search terms to use aren't obvious.

So in any data set, the lowest value would be the 0th permeantile, the highest would be the 100th and the mean would be the 50th.

And to find a particular permeantile, you have to find the point that would become the mean if you weight the data set in a certain way. For example, if you had a uniform distribution between 0 and 1, you'd want the 75th permeantile to be 0.75. And because you essentially have to take into account both "mass" and "distance" from the centre of mass, to make 0.75 the mean, you have to weight at a 9 to 1 ratio rather than 3 to 1 (9 being 3 squared).

So to find the pth permeantile, you find the point that becomes the mean if you weight everything below it by (100-p)^2 and weight everything above it by p^2.

So going back to the competitions, you simply find the permeantile for each score in each round, and these permeantiles would become the points that the competitors get. To take one example you get (permeantile in brackets after raw score):

1. 100 (100)

2. 99 (90.91) - http://www.wolframalpha.com/input/?i=(100*(p%5E2)+%2B+99+%2B+98*((100-p)%5E2)+%2B+0*((100-p)%5E2))+%2F+(2*((100-p)%5E2)+%2B+1+%2B+(p%5E2)+)+%3D+99

3. 98 (85.11) - http://www.wolframalpha.com/input/?i=(100*(p%5E2)+%2B+99*(p%5E2)+%2B+98+%2B+0*((100-p)%5E2))+%2F+(((100-p)%5E2)+%2B+1+%2B+2*(p%5E2)+)+%3D+98

4. 0 (0)