gnubg "luck rate"
Kees van den Doel
Whatever it is, it does not seem to reflect the intuitive notion of "the
part of the matchscore due to luck".
The winner of the match is always the "luckiest". I tested it by playing
some matches against gnubg 0 ply noise level 1.0. Of course I win a
gammon every game (it still knows to avoid bg's at this level) but in
the analysis my "luck" is always bigger than gnu's.
A better definition of "luck" would show that in such matches as
described above the "luck" evens out (the result being almost entirely
due to "skill").
Anyways, if someone would be kind enough to define "luck" in gnubg
analysis, one can try to improve on this definition. Something that captures
the intuitive notion of "skill" versus "luck".
Kees (Yours on some kicks for many as tapes for which inspired to much
asked, of 2-5 years ago, there are pointing arrowhead on earth.)
Adam Stocks
"Kees van den Doel" <kvan...@xs4all.nl> wrote in message
news:3ce87d1f$0$3876$e4fe...@dreader4.news.xs4all.nl...
> What's the definition of the gnubg "luck rate"?
The calculation of the luck rate is similar in principle to the calculation
of the error rate - that is, it is the difference in equity between two
'moves'. The error rate is the average amount of equity loss between the
'best move' and the actual move played. To calculate the luck rate, the bot
looks at the position before the dice are rolled, and works out the average
equity of the various positions resulting from each of the possible dice
rolls. Then after the roll, the equity of the 'best move' will be either
above or below that average. Doug Zare explains this in more detail in his
article 'A Measure of Luck' :
http://www.gammonvillage.com/backgammon/news/article_display.cfm?resourceid=
411
where he defines the luck rate: ' The mathematical measure of luck gained
on a roll of the dice is your equity after the roll minus your equity before
the roll, i.e., luck is the equity you gain through the roll of the dice. It
is the equity of your best possible play minus your equity before the roll,
or your equity after your opponent's best play minus your equity before the
roll. '
Snowie for instance tells you (on it's default settings) that you rolled a
'joker' when this equity difference is greater than 0.5 (i.e. you
effectively won half a game on one roll - you had a luck rate of +0.5 for
that roll. Of course, your opponent would have lost 0.5 equity, so your
good luck is, de facto, the same as your opponent's bad luck, and vice
versa.
> Whatever it is, it does not seem to reflect the intuitive notion of "the
> part of the matchscore due to luck".
>
> The winner of the match is always the "luckiest". I tested it by playing
> some matches against gnubg 0 ply noise level 1.0. Of course I win a
> gammon every game (it still knows to avoid bg's at this level) but in
> the analysis my "luck" is always bigger than gnu's.
For players of equal skill, the winner of each game needs a total luck of
+0.5 (EMG equity), because they both started off with an equal chance of 50%
(0.00 EMG equity), but the winner ends up with 100% (+1.00 EMG equity). In
a match of more than 1 point, the luck rates (and error rates) is usually
measured in Normalised EMG (Equivalent to Money Game) units, so the actual
figures you see will usually be much greater than +0.5 for each player.
However, it is not necessarily true that the winner of a game or match is
always the 'luckiest'. That is normally true when the players are faily
close in ability, but if the world class gnu (2(3)-ply, no noise) plays a
novice, the novice will still be the underdog even if he is significantly
luckier than gnu during the game/match.
> A better definition of "luck" would show that in such matches as
> described above the "luck" evens out (the result being almost entirely
> due to "skill").
You would need to play a large sample of matches to be statistically sure
that you weren't simply experiencing a run of good luck over a short series
of matches.
> Anyways, if someone would be kind enough to define "luck" in gnubg
> analysis, one can try to improve on this definition. Something that
captures
> the intuitive notion of "skill" versus "luck".
Adam
Kees van den Doel
Adam Stocks <ad...@stocks49.freeserve.co.uk> wrote:
>"Kees van den Doel" <kvan...@xs4all.nl> wrote in message
>news:3ce87d1f$0$3876$e4fe...@dreader4.news.xs4all.nl...
>> What's the definition of the gnubg "luck rate"?
Thanks for the explanation. It sounds obvious and right.
>http://www.gammonvillage.com/backgammon/news/article_display.cfm?resourceid=
>411
Hmm, I have to pay to read this?
>> Whatever it is, it does not seem to reflect the intuitive notion of "the
>> part of the matchscore due to luck".
>> The winner of the match is always the "luckiest". I tested it by playing
>> some matches against gnubg 0 ply noise level 1.0. Of course I win a
>> gammon every game (it still knows to avoid bg's at this level) but in
>> the analysis my "luck" is always bigger than gnu's.
>You would need to play a large sample of matches to be statistically sure
>that you weren't simply experiencing a run of good luck over a short series
>of matches.
Well, over a period of several months I've analyzed hundreds of matches
of various skill level differences and I've never ever encountered a
single one where the winner of the match did not receive the highest
"luck" rate. This is why I have some doubts about the meaningfulness of
the luck rate.
However I've been able to generate some example games between gnu 0 ply
noise 1.0 and gnu 2 ply where this does occur. Of course this is an
extreme case in skill difference.
Does anyone have any statistical empirical data to validate the claim
that this measure of luck corresponds to what we want to measure?
Would this experiment work?:
Play n games between A and B who's ratings are known, so we know the
expected fraction a_r of the n games won by A. Measure the actual
fraction won, a_c. If a_c < a_r the luck rate of A must come out lower
than the luck rate of B and the other way around. If it doesn't,
something is wrong with the definition.
Kees (Kees amen Article 4671 of new meaning less lucky fjew is confuus
en smeet hem gelezen heb duidelijk adrvertizing the connectivity
properties of normal in of existence for ever.)
PS. Is there a way to let gnubg play a match against itself without
these dialogues coming up after each games?
Peter Sochovsky
2ply, there is ALWAYS the same result: The winner was luckier.
I want to know the reason for this either.
Peter
Douglas Zare
Peter Sochovsky wrote:
Why is this a problem? It may bother you if you don't have the right model
of what happens in a backgammon match. It's not that both players roll a
die called luck, and whichever gets a higher value is the winner. What
happens is that you match winning chances follow a random walk with drift
depending on the skill level, with absorbing barriers at 0 and 1.
In matches between players of greatly different skill levels, the weaker
player will still win some matches. The weaker player needs a huge positive
amount of luck to win. The stronger player needs a small, positive amount
of luck to win. Luck averages to 0, hence the stronger player wins the vast
majority of the matches, on average.
Douglas Zare
Ian Shaw
"Douglas Zare" <za...@math.columbia.edu> wrote in message
news:3CE9E34E...@math.columbia.edu...
> What
> happens is that your match winning chances follow a random walk with drift
> depending on the skill level, with absorbing barriers at 0 and 1.
>
>
thing? Thanks for a mathematicians-eye view Douglas.
Ian
Adrian Pitt
wrote:
>
>Well, over a period of several months I've analyzed hundreds of matches
>of various skill level differences and I've never ever encountered a
>single one where the winner of the match did not receive the highest
>"luck" rate. This is why I have some doubts about the meaningfulness of
>the luck rate.
>
Well your extraordinarily lucky :-) I've played many matches on Snowie
where I was the lucky player and lost. Hardly surprising. Even with
my good rolling it stil beats me. I really dont see what all the fuss
is about. Snowie handles it correctly and handles it well. Being
lucky is only one variable in deciding who wins a match.
Adrian
Kees van den Doel
Adrian Pitt <ap...@nectar.com.au> wrote:
>>Well, over a period of several months I've analyzed hundreds of matches
>>of various skill level differences and I've never ever encountered a
>>single one where the winner of the match did not receive the highest
>>"luck" rate. This is why I have some doubts about the meaningfulness of
>>the luck rate.
>Well your extraordinarily lucky :-)
That's of course complete nonsense.
>I've played many matches on Snowie where I was the lucky player and
>lost. Hardly surprising. Even with my good rolling it stil beats me.
>I really dont see what all the fuss is about. Snowie handles it
>correctly and handles it well. Being lucky is only one variable in
>deciding who wins a match.
The fuss is about gnubg behaving differently (winner always luckiest).
Could it be as simple as a gnubg bug in computing the luck rate?
Kees (Kees bugs have two months in me, Behold, Milcah, the CIA, KGB,
FBI, NRA, GNU, IGA, ABC, NSA, DIA, etc.)
Adam Stocks
"Adam Stocks" <ad...@stocks49.freeserve.co.uk> wrote in message
news:acaoe3$89b$1...@newsg3.svr.pol.co.uk...
> For players of equal skill, the winner of each game needs a total luck of
> +0.5 (EMG equity), because.......
That should have read '...total luck of +1.00 (EMG equity)...' (=+0.5
MWC)
Adam
Kees van den Doel
Douglas Zare <za...@math.columbia.edu> wrote:
I didn't get this untill I found the free draft of your article on
http://www.math.columbia.edu/~zare/luckequity.html
With that definition of luck (call it "zluck") you can only win a match
without being zluckier if your opponent lost at least a whole point
match equity due to mistakes.
If would be interesting to be able to plot e(i), the match (or, properly
modified, the game-) equity as it evolved over the games (the random
walk, e(0)=1, e(last) = +-1). There would be 4 kinds of transitions due
to dice throws and mistakes by either parties and also cube decisions. I
think I screw up a lot when e() is large or small, which should be
obvious from looking at such a plot. I wonder if you could deduce the
game type from looking at such a plot (backgame, attacking game,
champ-goofy, etc.).
Any chance of a GNUBG developer warming to this idea?
Coming back to zluck; a nice definition but a bit different from how we
expect luck to behave perhaps. Let me try to define a more intuitive
quantity "iluck".
How should it behave? First of all, with perfect play on both sides the
iluckiest player will win. An amount of 0+ iluck will be needed. (Game
is even to start, so any amount of positive iluck will be enough to
win.) Likewise the iluck for the loser will be 0-. (I believe these
can be rigorously defined as infinitesimals, numbers that are infinitely
small positive and negative). If your relative skill S is defined as
your total equity gain due to mistakes (negative) minus your opponents
equity gain due to mistakes, your iluck will be -S if you win. The
losers iluck will be just +S.
I think this is a more intuitive definition, which sort of shows luck
does not exist (it's just the opposite of skill).
In terms of zluck for the winner we have i luck = zluck-1, for the loser
iluck = -zluck -1.
I think I will also stop saying "good luck" at the start of my matches
now, as it is sort of an insult. "I wish you an infinitemimal amount of
iluck", or "I wish you 1 zluck" would be politer, perhaps followed by a
quote of Douglas' article.
Kees (Then again, they feel safe your special status quo formele
gronden.)
Kees van den Doel
Kees van den Doel <kvan...@xs4all.nl> wrote:
>If would be interesting to be able to plot e(i), the match (or, properly
>modified, the game-) equity as it evolved over the games (the random
>walk, e(0)=1, e(last) = +-1). There would be 4 kinds of transitions due
>to dice throws and mistakes by either parties and also cube decisions. I
>think I screw up a lot when e() is large or small, which should be
>obvious from looking at such a plot. I wonder if you could deduce the
>game type from looking at such a plot (backgame, attacking game,
>champ-goofy, etc.).
>
>Any chance of a GNUBG developer warming to this idea?
I've made such a plot for a single game by hand (i.e., typing in the
equities and read it in MATLAB), admirable at
http://www.xs4all.nl/~kvandoel/bg
If I could just get GNUBG to dump the equity array into a txt file...
Kees (I think for temporary mental level to seize.)