Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Weighted value of skill required in various positions
116 views
Skip to first unread message
mu...@compuplus.net
Jan 21, 2015, 9:41:46 PM1/21/15
to
Another thing I got thinking about is the varying amount of
skill required to pick the best moves in various positions.
I understand that bots ignore forced moves in their skill
calculations because no skill is required for a forced move.
What about an "almost forced" move? Having only two possible
moves, for example?
Surely they wouldn't require as much skill as positions with
ten possible moves. Or would they?
Consider these positions with: only two close moves, only two
distant moves, ten moves with four close bests, ten moves with
one clear best, etc... Which ones would require more skill to
pick?
Then combine those with when they occur within the game. How
critical (i.e. how skill valuable) is it to pick the best one
of only two moves in the early/middle/late stages of a game?
Are the current bots even trying to tackle such complexities
in their otherwise quite pretentious looking calculations?
Too much to worry about? Too difficult to implement?? Any
other thoughts???
MK
skill required to pick the best moves in various positions.
I understand that bots ignore forced moves in their skill
calculations because no skill is required for a forced move.
What about an "almost forced" move? Having only two possible
moves, for example?
Surely they wouldn't require as much skill as positions with
ten possible moves. Or would they?
Consider these positions with: only two close moves, only two
distant moves, ten moves with four close bests, ten moves with
one clear best, etc... Which ones would require more skill to
pick?
Then combine those with when they occur within the game. How
critical (i.e. how skill valuable) is it to pick the best one
of only two moves in the early/middle/late stages of a game?
Are the current bots even trying to tackle such complexities
in their otherwise quite pretentious looking calculations?
Too much to worry about? Too difficult to implement?? Any
other thoughts???
MK
michae...@gmail.com
Jan 22, 2015, 7:56:00 AM1/22/15
to
I mean they don't put it on scale and say wow, you got 10 difficult moves to think of, you got it right, here are 10 kilos of skill for you ;-)
They simply deduct "skill points" from you for every deviation you get from best move.In fact suppose you played just one roll and from there on you stayed on bar (you made no mistakes whatsoever it may even rank you supernatural !!
This is more funnily obvious on XG. During my 15 days of trial I 've passed it some 30 of matches to analyze. And wow, i was so thrilled and happy while at the beginning of every match analysis it would rank me "World Class". Slowly slowly it would drop me down though ;-))))
Walt
Jan 22, 2015, 8:38:19 AM1/22/15
to
On 1/22/2015 7:55 AM, michae...@gmail.com wrote:
>
> The bots don't measure skill the way you think...
> In fact suppose you played just one roll and from there on you stayed on bar (you made no mistakes whatsoever it may even rank you supernatural !!
Snowie did it that way, and as you imply it makes no sense.
XG does not do it that way, and Xavier has deliberately not implemented
the Snowie ER metric in XG. Here's why:
Can I see the Snowie Rating?
No. Our take on the Snowie Rating is that it is inherently flawed.
Snowie rating is the amount of equity (normalized) lost multiply by 1000
divided by the number for roll (for both players). The main problem of
it is that it does not take into account forced moves or moves that are
completely unimportant. For instance in Snowie, if you play a game and
reach a position where are 100% sure to lose and play the next 10 moves
rather than resigning your rating will be higher! That surely does not
make sense. Another case where the result is not proper is if your
opponent makes a big mistake at the beginning of the game, get hit and
dance for pretty much the rest of the game. His Snowie Rating will be
very low because of all the moves you made and was sitting on the bar
obviously making no error. But, after all, he was given one chance to
show your skills but made a bad choice. In my book, he played bad every
move and should be rated very poorly on that game. We are committed to
make eXtreme the standard in Backgammon and bringing back old system
that we don't believe in is not something we want to do.
eXtreme Gammon has the ability to display the Performance Rating. It is
the equity lost per decision multiplied by 500. So the number are in the
range of Snowie ER, but still take into account that force move are not
to be counted.
What constitute a decision?
A decision is a checker move or a cube double that is considered
non-obvious by the computer. Considered obvious are the following:
- A forced moved
- A move for which the best choice and the worst choice have an equity
difference of less than 0.001.
- A double action where the equity before doubling and after doubling is
the same (optional double)
- A double action where the equity before doubling is 0.200 larger than
the equity after the double (obvious non double)
- A double action where the equity before doubling is 0.200 larger than
the equity after a drop (obvious too good to double)
- A double action where the equity before and after a double is very
negative (This corresponds to a loss of position when trailing in the
match by a lot)
Are never consider as a obvious:
- A double decision resulting in a double
- A take decision
See https://www.extremegammon.com/support.aspx
>
> The bots don't measure skill the way you think...
> In fact suppose you played just one roll and from there on you stayed on bar (you made no mistakes whatsoever it may even rank you supernatural !!
XG does not do it that way, and Xavier has deliberately not implemented
the Snowie ER metric in XG. Here's why:
Can I see the Snowie Rating?
No. Our take on the Snowie Rating is that it is inherently flawed.
Snowie rating is the amount of equity (normalized) lost multiply by 1000
divided by the number for roll (for both players). The main problem of
it is that it does not take into account forced moves or moves that are
completely unimportant. For instance in Snowie, if you play a game and
reach a position where are 100% sure to lose and play the next 10 moves
rather than resigning your rating will be higher! That surely does not
make sense. Another case where the result is not proper is if your
opponent makes a big mistake at the beginning of the game, get hit and
dance for pretty much the rest of the game. His Snowie Rating will be
very low because of all the moves you made and was sitting on the bar
obviously making no error. But, after all, he was given one chance to
show your skills but made a bad choice. In my book, he played bad every
move and should be rated very poorly on that game. We are committed to
make eXtreme the standard in Backgammon and bringing back old system
that we don't believe in is not something we want to do.
eXtreme Gammon has the ability to display the Performance Rating. It is
the equity lost per decision multiplied by 500. So the number are in the
range of Snowie ER, but still take into account that force move are not
to be counted.
What constitute a decision?
A decision is a checker move or a cube double that is considered
non-obvious by the computer. Considered obvious are the following:
- A forced moved
- A move for which the best choice and the worst choice have an equity
difference of less than 0.001.
- A double action where the equity before doubling and after doubling is
the same (optional double)
- A double action where the equity before doubling is 0.200 larger than
the equity after the double (obvious non double)
- A double action where the equity before doubling is 0.200 larger than
the equity after a drop (obvious too good to double)
- A double action where the equity before and after a double is very
negative (This corresponds to a loss of position when trailing in the
match by a lot)
Are never consider as a obvious:
- A double decision resulting in a double
- A take decision
See https://www.extremegammon.com/support.aspx
michae...@gmail.com
Jan 22, 2015, 9:10:51 AM1/22/15
to
On Thursday, January 22, 2015 at 3:38:19 PM UTC+2, Walt wrote:
>
> XG does not do it that way, and Xavier has deliberately not implemented
> the Snowie ER metric in XG. Here's why:
OK XG does not take into account forced moves, but still the way it measures skill is by deducting rating points according to errors. That's the reason at the beginning of analyzing a match it shows everyone as "world class".
>
> XG does not do it that way, and Xavier has deliberately not implemented
> the Snowie ER metric in XG. Here's why:
Walt
Jan 22, 2015, 10:09:24 AM1/22/15
to
decisions to get an error rate. Is there a better way?
BTW, gnubg does something similar - it does not count forced or obvious
moves like snowie did. Not exactly sure of the details.
--
//Walt
michae...@gmail.com
Jan 22, 2015, 2:06:01 PM1/22/15
to
On Thursday, January 22, 2015 at 5:09:24 PM UTC+2, Walt wrote:
>
>
> Right. It adds up the equity losses and then divides by the number of
> decisions to get an error rate. Is there a better way?
>
> BTW, gnubg does something similar - it does not count forced or obvious
> moves like snowie did. Not exactly sure of the details.
> --
> //Walt
We agree. Now read Murat's post and tell me if this is how he meant it
>
>
> Right. It adds up the equity losses and then divides by the number of
> decisions to get an error rate. Is there a better way?
>
> BTW, gnubg does something similar - it does not count forced or obvious
> moves like snowie did. Not exactly sure of the details.
> --
> //Walt
Walt
Jan 22, 2015, 5:14:47 PM1/22/15
to
On 1/22/2015 2:06 PM, michae...@gmail.com wrote:
> Now read Murat's post and tell me if this is how he meant it
Sorry, but I gave up on that many years ago.
> Now read Murat's post and tell me if this is how he meant it
--
//Walt
Tim Chow
Jan 22, 2015, 8:52:24 PM1/22/15
to
On Wednesday, January 21, 2015 at 9:41:46 PM UTC-5, mu...@compuplus.net wrote:
> Another thing I got thinking about is the varying amount of
> skill required to pick the best moves in various positions.
>
> I understand that bots ignore forced moves in their skill
> calculations because no skill is required for a forced move.
True for GNU and XG, but not Snowie.
> Another thing I got thinking about is the varying amount of
> skill required to pick the best moves in various positions.
>
> I understand that bots ignore forced moves in their skill
> calculations because no skill is required for a forced move.
> What about an "almost forced" move? Having only two possible
> moves, for example?
> Surely they wouldn't require as much skill as positions with
> ten possible moves. Or would they?
>
> Consider these positions with: only two close moves, only two
> distant moves, ten moves with four close bests, ten moves with
> one clear best, etc... Which ones would require more skill to
> pick?
>
> Then combine those with when they occur within the game. How
> critical (i.e. how skill valuable) is it to pick the best one
> of only two moves in the early/middle/late stages of a game?
>
> Are the current bots even trying to tackle such complexities
> in their otherwise quite pretentious looking calculations?
What makes this question difficult, in my opinion, is that if we define skill to mean something like the amount of understanding of backgammon that is needed to select the right play, then the amount of skill does not correlate very well, if at all, with the number of options or how close they are. If all I know is how many options there are and how close or far apart they are, that tells me almost nothing about how hard it is to pick the best play.
For example, suppose there is no contact left and we're both just bearing off. I roll 11, and I have four checkers on the ace point. Everyone knows to bear off four checkers. This is an easy decision even though there might theoretically be dozens of legal options.
Maybe what makes it easy is that the best play is far ahead of the other plays? But suppose that in the aforementioned race, I'm way ahead in the race, and 99.9% to win anyway. Then perhaps bearing off three checkers is only microscopically worse than bearing off four checkers. It's still an easy decision.
And of course we've all faced decisions where there are only two legal moves yet it is very puzzling which move is better. Sometimes the pros and cons of the two plays balance out almost exactly while sometimes one play is much better, but it's difficult to tell.
---
Tim Chow
Tim Chow
Jan 22, 2015, 9:01:42 PM1/22/15
to
On Thursday, January 22, 2015 at 8:38:19 AM UTC-5, Walt wrote:
---
Tim Chow
> On 1/22/2015 7:55 AM, michael wrote:
> > In fact suppose you played just one roll and from there on you stayed on
> > bar (you made no mistakes whatsoever it may even rank you supernatural !!
>
> Snowie did it that way, and as you imply it makes no sense.
For what it's worth, I don't agree that it makes no sense. Skill measures only make sense when you average over the long term anyway. If you look only at the short term, then both schemes have problems. If we adopt the XG approach, imagine that you played just one roll but played it "incorrectly" and then stayed on the bar. XG may then rate you a Beginner or even Distracted for that tiny alleged "error."
> > In fact suppose you played just one roll and from there on you stayed on
> > bar (you made no mistakes whatsoever it may even rank you supernatural !!
>
> Snowie did it that way, and as you imply it makes no sense.
---
Tim Chow
mu...@compuplus.net
Jan 23, 2015, 12:06:58 PM1/23/15
to
On Thursday, January 22, 2015 at 6:52:24 PM UTC-7, Tim Chow wrote:
> On Wednesday, January 21, 2015 at 9:41:46 PM UTC-5, mu...@compuplus.net wrote:
>> Then combine those with when they occur within the game. How
>> critical (i.e. how skill valuable) is it to pick the best one
>> of only two moves in the early/middle/late stages of a game?
>>
>> Are the current bots even trying to tackle such complexities
>> in their otherwise quite pretentious looking calculations?
> Only in the sense that I described above, so not really.
> .........
> On Wednesday, January 21, 2015 at 9:41:46 PM UTC-5, mu...@compuplus.net wrote:
>> Then combine those with when they occur within the game. How
>> critical (i.e. how skill valuable) is it to pick the best one
>> of only two moves in the early/middle/late stages of a game?
>>
>> Are the current bots even trying to tackle such complexities
>> in their otherwise quite pretentious looking calculations?
> Only in the sense that I described above, so not really.
> And of course we've all faced decisions where there are only
> two legal moves yet it is very puzzling which move is better.
With the simpler example of two possible moves, I would think
> two legal moves yet it is very puzzling which move is better.
that they would not occur as often or be puzzling early in the
game than in the middle stages, and even much less towards the
end??
BTW, I'm not really interested in improving bots to calculate
skill/luck/error rate/etc. more accurately (which are obviously
worthless since nobody is willing bet money on them... ;)
MK
Tim Chow
Jan 23, 2015, 7:39:06 PM1/23/15
to
On Friday, January 23, 2015 at 12:06:58 PM UTC-5, mu...@compuplus.net wrote:
> With the simpler example of two possible moves, I would think
> that they would not occur as often or be puzzling early in the
> game than in the middle stages, and even much less towards the
> end??
I agree that it is rare in the early stages of the game to have only two legal ways to play a move.
> With the simpler example of two possible moves, I would think
> that they would not occur as often or be puzzling early in the
> game than in the middle stages, and even much less towards the
> end??
But I think it occurs fairly often towards the end of the game. The first example that comes to mind is that I am bearing off with my opponent on the bar. I roll two high numbers and am forced to leave a blot. One option bears off an extra checker but gives the opponent slightly more rolls to hit compared to the other option. I find that these decisions can be quite tricky.
---
Tim Chow
0 new messages