Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Error rate, elo, odds and ends of winning...
93 views
Skip to first unread message
mu...@compuplus.net
Nov 24, 2014, 11:28:09 PM11/24/14
to
On Monday, November 24, 2014 6:25:15 PM UTC-7, Tim Chow wrote:
> On Monday, November 24, 2014 2:56:07 PM UTC-5, mu...@compuplus.net wrote:
>> Clarify for me. Are you saying that odds calculations using
>> "error rates" are just hocus-pocus even though the "error
>> rates" themselves are not? Or are you saying that odds
>> calculations are just hocus-pocus because the "error rates"
>> used are hocus-pocus themselves??
> I wouldn't say that "error rates are hocus-pocus themselves"
> because error rates can be useful when interpreted cautiously
> and applied in the right way.
Since you seem to have made a reasonable effort to respond, I
started a new thread with the hope of having a productive
discussion.
> When the computer recommends Play A and flags Play B as an
> error, it just means that if the rest of the game or match
> were played out by computers on both sides, then Play A would
> lead to better results. This doesn't mean, of course, that
> if the rest of the game or match were played out with a human
> on one side or both sides, then Play A would lead to better
> results. So if your goal is better results (and what other
> goal would make sense?), always going with the computer play
> isn't necessarily the right strategy.
I'm surprised that you would actually say something like this
and I honestly don't understand why or how it matters whether
both sides are bots or not.
A human playing differently than the bot would cause future
moves to branch out differently but that would be taken care
of by a roll-out, no?
From all the position discussions that you guys have without
end, I had the impression that at their strongest settings
bots play practically at roll-out level of accuracy..??
What you are saying above could only make sense to me if you
were inferring that bots can be fundamentally wrong in their
way of picking the best moves, making roll-outs completely
bot-subjective and pretty worthless...
Otherwise, I can't see how a certain play during the course
of a game is made by a bot or human can make a difference.
> However, when the computer flags Play B as a big error, it's
> usually possible to discern why. In many cases, once the
> "why" is understood, one can see that Play B would be a
> mistake against most players, and not just a mistake when a
> bot plays a bot.
I suppose this goes towards your argument that odds of winning
is only a guess, an approximation??
If so, I wouldn't have a problem with it at all and all I would
ask from you then, would be to tell me just how inaccurate or
approximate the derived odds would be?
And then, of course, tell me whether you would stand behind that
"lower end of the approximation"?? :)
> One can also sometimes discern the circumstances under which
> Play B would actually be a good play.
You are confusing me again. You don't mean that bots can do this,
do you?
In other words, you don't mean to say that bots can knowingly
blunder because they can discern that it's actually a good (if
not a better) play under some circumstances?
I wonder if you are coming around to agree that humans are still
superior to bots because they can do this while bots cant..?? ;)
> So, the computer's notion of an error is correlated with, but
> not exactly aligned with, "true" errors (i.e., the wrong play
> when all circumstances are taken into account, including who
> is playing whom, psychological factors, etc.).
No need to complicated any more than necessary. Let's just focus
on a human playing against the best of the bots.
> Because it is correlated, it is a useful tool for improving
> one's understanding of the game, provided one does not trust
> it blindly but uses it as a guide.
Okay, so now you are back to your normal self. :) "Improving"
here meaning becoming a more bot-like thus better player...
You may negate me but I just don't think you are (generally)
talking about a human learning from the bot's mistakes.
> But because it is not exactly aligned, taking it too literally
> and using it to calculate odds is going to lead to unreliable
> results.
Okay, fine, as I said before, I will be willing to hear what
would be your "safe" estimates?
If you don't agree with 1 ER = 33 ELO, 200 ELO difference 66%
favorite to win, etc. what would you propose as not complete
hocus-pocus but reasonable/safe numbers and stand behind..??
MK
> On Monday, November 24, 2014 2:56:07 PM UTC-5, mu...@compuplus.net wrote:
>> Clarify for me. Are you saying that odds calculations using
>> "error rates" are just hocus-pocus even though the "error
>> rates" themselves are not? Or are you saying that odds
>> calculations are just hocus-pocus because the "error rates"
>> used are hocus-pocus themselves??
> I wouldn't say that "error rates are hocus-pocus themselves"
> because error rates can be useful when interpreted cautiously
> and applied in the right way.
Since you seem to have made a reasonable effort to respond, I
started a new thread with the hope of having a productive
discussion.
> When the computer recommends Play A and flags Play B as an
> error, it just means that if the rest of the game or match
> were played out by computers on both sides, then Play A would
> lead to better results. This doesn't mean, of course, that
> if the rest of the game or match were played out with a human
> on one side or both sides, then Play A would lead to better
> results. So if your goal is better results (and what other
> goal would make sense?), always going with the computer play
> isn't necessarily the right strategy.
I'm surprised that you would actually say something like this
and I honestly don't understand why or how it matters whether
both sides are bots or not.
A human playing differently than the bot would cause future
moves to branch out differently but that would be taken care
of by a roll-out, no?
From all the position discussions that you guys have without
end, I had the impression that at their strongest settings
bots play practically at roll-out level of accuracy..??
What you are saying above could only make sense to me if you
were inferring that bots can be fundamentally wrong in their
way of picking the best moves, making roll-outs completely
bot-subjective and pretty worthless...
Otherwise, I can't see how a certain play during the course
of a game is made by a bot or human can make a difference.
> However, when the computer flags Play B as a big error, it's
> usually possible to discern why. In many cases, once the
> "why" is understood, one can see that Play B would be a
> mistake against most players, and not just a mistake when a
> bot plays a bot.
I suppose this goes towards your argument that odds of winning
is only a guess, an approximation??
If so, I wouldn't have a problem with it at all and all I would
ask from you then, would be to tell me just how inaccurate or
approximate the derived odds would be?
And then, of course, tell me whether you would stand behind that
"lower end of the approximation"?? :)
> One can also sometimes discern the circumstances under which
> Play B would actually be a good play.
You are confusing me again. You don't mean that bots can do this,
do you?
In other words, you don't mean to say that bots can knowingly
blunder because they can discern that it's actually a good (if
not a better) play under some circumstances?
I wonder if you are coming around to agree that humans are still
superior to bots because they can do this while bots cant..?? ;)
> So, the computer's notion of an error is correlated with, but
> not exactly aligned with, "true" errors (i.e., the wrong play
> when all circumstances are taken into account, including who
> is playing whom, psychological factors, etc.).
No need to complicated any more than necessary. Let's just focus
on a human playing against the best of the bots.
> Because it is correlated, it is a useful tool for improving
> one's understanding of the game, provided one does not trust
> it blindly but uses it as a guide.
Okay, so now you are back to your normal self. :) "Improving"
here meaning becoming a more bot-like thus better player...
You may negate me but I just don't think you are (generally)
talking about a human learning from the bot's mistakes.
> But because it is not exactly aligned, taking it too literally
> and using it to calculate odds is going to lead to unreliable
> results.
Okay, fine, as I said before, I will be willing to hear what
would be your "safe" estimates?
If you don't agree with 1 ER = 33 ELO, 200 ELO difference 66%
favorite to win, etc. what would you propose as not complete
hocus-pocus but reasonable/safe numbers and stand behind..??
MK
Tim Chow
Nov 25, 2014, 8:55:47 AM11/25/14
to
On Monday, November 24, 2014 11:28:09 PM UTC-5, mu...@compuplus.net wrote:
> A human playing differently than the bot would cause future
> moves to branch out differently but that would be taken care
> of by a roll-out, no?
The simplest example is this: There can be positions where the player on roll should not double against a bot, but where a human opponent might drop the cube. Here's an example:
> A human playing differently than the bot would cause future
> moves to branch out differently but that would be taken care
> of by a roll-out, no?
XGID=-ABC-------------------bc-:1:1:1:00:0:0:0:0:10
X:Player 1 O:Player 2
Score is X:0 O:0. Unlimited Game
+13-14-15-16-17-18------19-20-21-22-23-24-+
| | | O O |
| | | O O |
| | | O |
| | | |
| | | |
| |BAR| |
| | | |
| | | |
| | | X | +---+
| | | X X | | 2 |
| | | X X X | +---+
+12-11-10--9--8--7-------6--5--4--3--2--1-+
Pip count X: 14 O: 7 X-O: 0-0
Cube: 2, X own cube
X on roll, cube action
I know many people who, playing O, would definitely drop if I were to double here. Doubling is therefore the correct play against them. A bot, however, would not double as X here and would take as O. If you were to play this position hundreds of times against the bot, doubling every time and playing the position out to the end, and if I were to sit next to you playing the same position against the bot the same number of times, but religiously following the bot's advice about how to play (and in particular not doubling initially), then I would expect to win significantly more points than you would. So that is why I say that doubling is correct against most humans but incorrect against the bot.
> What you are saying above could only make sense to me if you
> were inferring that bots can be fundamentally wrong in their
> way of picking the best moves, making roll-outs completely
> bot-subjective and pretty worthless...
The rollouts are "bot-subjective" but still not worthless. They tell me not to double against Neil. That is useful information. The bots cannot, of course, tell me to double in my home chouette. (Though XG will suggest that if there is an 8.7% chance that my opponent will drop then I should double. The 8.7% is again not gospel because it assumes that subsequent cube decisions will be made the way the bot would, which might not be true.)
> Otherwise, I can't see how a certain play during the course
> of a game is made by a bot or human can make a difference.
> If so, I wouldn't have a problem with it at all and all I would
> ask from you then, would be to tell me just how inaccurate or
> approximate the derived odds would be?
>
> And then, of course, tell me whether you would stand behind that
> "lower end of the approximation"?? :)
> > One can also sometimes discern the circumstances under which
> > Play B would actually be a good play.
>
> You are confusing me again. You don't mean that bots can do this,
> do you?
> In other words, you don't mean to say that bots can knowingly
> blunder because they can discern that it's actually a good (if
> not a better) play under some circumstances?
> I wonder if you are coming around to agree that humans are still
> superior to bots because they can do this while bots cant..?? ;)
> > Because it is correlated, it is a useful tool for improving
> > one's understanding of the game, provided one does not trust
> > it blindly but uses it as a guide.
>
> Okay, so now you are back to your normal self. :) "Improving"
> here meaning becoming a more bot-like thus better player...
> You may negate me but I just don't think you are (generally)
> talking about a human learning from the bot's mistakes.
> Okay, fine, as I said before, I will be willing to hear what
> would be your "safe" estimates?
>
> If you don't agree with 1 ER = 33 ELO, 200 ELO difference 66%
> favorite to win, etc. what would you propose as not complete
> hocus-pocus but reasonable/safe numbers and stand behind..??
---
Tim Chow
michae...@gmail.com
Nov 25, 2014, 12:07:58 PM11/25/14
to
On Tuesday, November 25, 2014 6:28:09 AM UTC+2, mu...@compuplus.net wrote:
> I had the impression that at their strongest settings
> bots play practically at roll-out level of accuracy..??
I use GnuBg, and the answer is no. It plays at evaluation level. It often happens that it changes it's mind after doing roll outs
> I had the impression that at their strongest settings
> bots play practically at roll-out level of accuracy..??
> I suppose this goes towards your argument that odds of winning
> is only a guess, an approximation??
If you mean the odds of winning by playing a certain roll differently during a match,THAT IS an approximation.
> If so, I wouldn't have a problem with it at all and all I would
> ask from you then, would be to tell me just how inaccurate or
> approximate the derived odds would be?
> Okay, so now you are back to your normal self. :) "Improving"
> here meaning becoming a more bot-like thus better player...
For what is worth though there have been cases over the Internet that World class players disagreed with the bots.
> If you don't agree with 1 ER = 33 ELO, 200 ELO difference 66%...
What ER are you referring to? Snowie ER is different than GNUbg. Furthermore 0.5 Checker_mEMG + 0.5 Cube_mEMG in GNU in a 7 point match would not cost you 33 ELO points but 6.2+0.4=6.6
Here's the formula
FIBS abs rating in GNU
FIBSr =2050- (checker(N)*Checker_mEMG+Cube(N)*Cube_mEMG)
N=Match length
checker(N) = 8.798 + 25.526/N,
Cube(N) = 0.863 -0. 519/N.
Notice the formula itself has been questioned many times, make a search over the net, there are people who insist it is wrong especially on the cube(N) side.
michae...@gmail.com
Nov 25, 2014, 12:29:56 PM11/25/14
to
Also the winning probability depends on the match length N.
Assuming ELO diff=200
Then at N=21 it is 74%, at 7 it is 64% at 1 it is 55%
Here's the formula
Winning Probability=1-(1/(10^((ELO diff)*SQRT(N)/2000)+1))
Assuming ELO diff=200
Then at N=21 it is 74%, at 7 it is 64% at 1 it is 55%
Here's the formula
Winning Probability=1-(1/(10^((ELO diff)*SQRT(N)/2000)+1))
Tim Chow
Nov 25, 2014, 8:51:50 PM11/25/14
to
On Tuesday, November 25, 2014 12:07:58 PM UTC-5, michae...@gmail.com wrote:
> > I suppose this goes towards your argument that odds of winning
> > is only a guess, an approximation??
>
> In the example that Tim provided, no the odds are not an approximation,
> they can be calculated with mathematical accuracy.
In that example, the odds of *bot versus bot* can be calculated with mathematical accuracy. But the odds of *human versus bot* or *human versus human* can't be, because there's no mathematical formula that tells you the exact probability that the human will misplay something (usually a subsequent cube decision), even if you know what the human's "typical" error rate is.
> > I suppose this goes towards your argument that odds of winning
> > is only a guess, an approximation??
>
> In the example that Tim provided, no the odds are not an approximation,
> they can be calculated with mathematical accuracy.
> I personally don't like playing with bots. I only use them as a tool to
> improve my play. Whenever I come across a move (from an actual human vs
> human match) that the bot disagrees I try to spot the reasons. I am glad
> to say that I never failed in spotting the reasons, and never found a case
> where the bot was wrong.
And if you've never found a case where the bot was wrong, then you clearly haven't watched the bot try to roll a 15-checker prime all the way around the board from the opponent's home board to its own, and completely misunderstand the cube action along the way.
---
Tim Chow
michae...@gmail.com
Nov 26, 2014, 2:19:38 PM11/26/14
to
On Wednesday, November 26, 2014 3:51:50 AM UTC+2, Tim Chow wrote:
> > > I suppose this goes towards your argument that odds of winning
> > > is only a guess, an approximation??
> >
> > In the example that Tim provided, no the odds are not an approximation,
> > they can be calculated with mathematical accuracy.
>
> In that example, the odds of *bot versus bot* can be calculated with mathematical accuracy. But the odds of *human versus bot* or *human versus human* can't be, because there's no mathematical formula that tells you the exact probability that the human will misplay something (usually a subsequent cube decision), even if you know what the human's "typical" error rate is.
>
Yes of course but I think MK's question concerned the "odds of winning" as calculated by the bots.
> > > I suppose this goes towards your argument that odds of winning
> > > is only a guess, an approximation??
> >
> > In the example that Tim provided, no the odds are not an approximation,
> > they can be calculated with mathematical accuracy.
>
> In that example, the odds of *bot versus bot* can be calculated with mathematical accuracy. But the odds of *human versus bot* or *human versus human* can't be, because there's no mathematical formula that tells you the exact probability that the human will misplay something (usually a subsequent cube decision), even if you know what the human's "typical" error rate is.
>
> > I personally don't like playing with bots. I only use them as a tool to
> > improve my play. Whenever I come across a move (from an actual human vs
> > human match) that the bot disagrees I try to spot the reasons. I am glad
> > to say that I never failed in spotting the reasons, and never found a case
> > where the bot was wrong.
>
> There are plenty of positions where to this day I don't think I understand the reasons. I can, of course, always invent a story to "explain" the bot's choice, but that is not at all the same as understanding the true reasons. I can invent a story to explain why almost any play is right, even when it's not right.
I don't need to invent a whole story for the reasons.In about 80% of the cases the reasons are obvious, on 19% they become apparent after checking how all rolls play and there is about 1% that needs two or 3 ply tactical analysis which gets too complicated to do manually.
Tim Chow
Nov 26, 2014, 5:52:38 PM11/26/14
to
On Wednesday, November 26, 2014 2:19:38 PM UTC-5, michae...@gmail.com wrote:
> Yes of course but I think MK's question concerned the "odds of winning" as
> calculated by the bots.
No.
> Yes of course but I think MK's question concerned the "odds of winning" as
> calculated by the bots.
His concern is with the "odds of winning" when a *human plays a bot*, not with the "odds of winning" when a *bot plays a bot*. That is, he's interested not in the accuracy of the equity estimates, but in the accuracy of the Elo-like formulas that are commonly used to compute the odds when players of differing skill levels play each other.
---
Tim Chow
mu...@compuplus.net
Nov 27, 2014, 6:30:59 AM11/27/14
to
On Tuesday, November 25, 2014 6:55:47 AM UTC-7, Tim Chow wrote:
> On Monday, November 24, 2014 11:28:09 PM UTC-5, mu...@compuplus.net wrote:
>> A human playing differently than the bot would cause future
>> moves to branch out differently but that would be taken care
>> of by a roll-out, no?
> The simplest example is this: There can be positions where
> the player on roll should not double against a bot, but where
> a human opponent might drop the cube. Here's an example:
You have a very conditioned, "believer's" mind and thus you
> On Monday, November 24, 2014 11:28:09 PM UTC-5, mu...@compuplus.net wrote:
>> A human playing differently than the bot would cause future
>> moves to branch out differently but that would be taken care
>> of by a roll-out, no?
> The simplest example is this: There can be positions where
> the player on roll should not double against a bot, but where
> a human opponent might drop the cube. Here's an example:
can't understand any question asking you something different
than whatever stuff that you are already convinced of.
I'm talking about a human playing against a bot, in a way
different than a bot would play against itself. So, let's
look at your example with that in mind.
> XGID=-ABC-------------------bc-:1:1:1:00:0:0:0:0:10
>
> X:Player 1 O:Player 2
> Score is X:0 O:0. Unlimited Game
> +13-14-15-16-17-18------19-20-21-22-23-24-+
> | | | O O |
> | | | O O |
> | | | O |
> | | | |
> | | | |
> | |BAR| |
> | | | |
> | | | |
> | | | X | +---+
> | | | X X | | 2 |
> | | | X X X | +---+
> +12-11-10--9--8--7-------6--5--4--3--2--1-+
> Pip count X: 14 O: 7 X-O: 0-0
> Cube: 2, X own cube
> X on roll, cube action
your cube decision examples during final stages of bear off,
because you can't do any better than that and second because
that's basically the scope of the so-called "cube skill"...!
> I know many people who, playing O, would definitely drop if I
> were to double here. Doubling is therefore the correct play
> against them. A bot, however, would not double as X here and
> would take as O.
(against another human or a bot), but a bot would not. So,
since the bot would eliminate doubling here as an inferior
move, it would never make it against a human, another bot
or itself because it can't know who is playing against it.
> If you were to play this position hundreds of times against the
> bot, doubling every time and playing the position out to the end,
> and if I were to sit next to you playing the same position against
> the bot the same number of times, but religiously following the
> bot's advice about how to play (and in particular not doubling
> initially), then I would expect to win significantly more points
> than you would.
is only one notch above a coin toss in terms of simplicity.
If you were to look at other positions during the early and
middle stages of the game, the likelihood of your expectation
coming true would diminish and you would become less and less
capable of making such predictions. This is one of my points.
> The rollouts are "bot-subjective" but still not worthless. They
> tell me not to double against Neil.
is irrelevant to my argument/question, which basically boils
down to asking "Can you beat the bots making moves that are
considered inferior by the bot?", thus question the ability
of the bots in determining what moves are inferior/superior.
>> Otherwise, I can't see how a certain play during the course
>> of a game is made by a bot or human can make a difference.
> I picked a doubling decision for simplicity but similar principles
> apply to checker plays.
decision during the final rolls of a game because that's
all you are capable of... :( Next time, try to do a little
better.
>> And then, of course, tell me whether you would stand behind that
>> "lower end of the approximation"?? :)
> I don't know the answer to this question ahead of time. It would
> vary from person to person.
Fine, just pick a person with a certain error rate according
to a certain bot. Now answer these questions: what would your
expectations be about that person's chances of beating that bot,
how accurate would be your expectations and how accurate the
error rate or any other numbers you use in arriving at your
expectation? Is it really that difficult for you folks to
understand what I am asking? If somebody understands it and
notices that I am not asking it in a way generally understable
to all, can he please translate it for everybody...
> I would, however, stand behind the same "lower end of the
> approximation" that I have stated before. Namely, I believe
> that XGR+ would beat any human in a long series of, say,
> 11-point matches with at least 50% probability.
It's pathetic that you can't do any better to stand behind a bot
that you worship as the strongest on the planet. :((
And why not GBUBG? Why do you keep promoting eXtremeGarbage+ if
you can't even stand behind what supposedly makes it stronger??
>> In other words, you don't mean to say that bots can knowingly
>> blunder because they can discern that it's actually a good (if
>> not a better) play under some circumstances?
> I don't mean that. They cannot knowingly blunder. At least,
> current bots can't.
opponents error rate, etc. and begin to notice that it/he
is weaker, and start making wrong doubling decisions like
the one you gave above, because then its opponent would be
likely to drop and the bot would have purposefully turned
an inferior move into a superior move... (??)
But, for this, first you guys have to propose that there is
"continuity in the game" and be able to analyze beyond each
individual position separately, as it is done now.
I made this argument before but apparently nothing registers
with you folks beyond your own concocted theories... :(
>> I wonder if you are coming around to agree that humans are still
>> superior to bots because they can do this while bots cant..?? ;)
> Humans can do this while bots cannot, but unfortunately humans
> make too many other mistakes when playing.
that I am questioning whether a mistake according to the bot may
actually be the right move against that bot...!
The bot may never know what hit it :))
>>> Because it is correlated, it is a useful tool for improving
>>> one's understanding of the game, provided one does not trust
>>> it blindly but uses it as a guide.
>> Okay, so now you are back to your normal self. :) "Improving"
>> here meaning becoming a more bot-like thus better player...
> I didn't say that. By improving, I meant getting better results,
> whether playing against a bot or against a human. This often
> means becoming more bot-like but not always.
You keep wiggling with words like "often" but "not always", etc.
And I am patiently trying to pin you down on "how often?", "how
much not always?", etc. Frankly, the only I hope I have at this
point is that maybe somebody else reading our discussion will
be able to answer those and have to balls to admit what may not
add up...
>> Okay, fine, as I said before, I will be willing to hear what
>> would be your "safe" estimates?
>> If you don't agree with 1 ER = 33 ELO, 200 ELO difference 66%
>> favorite to win, etc. what would you propose as not complete
>> hocus-pocus but reasonable/safe numbers and stand behind..??
> I've already said that I would stand behind the statement that
> XGR+ plays 11-point matches better than a human does. Beyond
> that I have no proposal to offer.
about the accuracy of error rate, ELO, winning chances/odds, etc.
as calculated by the bots.
In fact, I ma not sure if you can even defend the soundness of
the concepts and the calculations/formulas themselves.
Could it be that they are simply elaborate bullshit...?? ;)
MK
mu...@compuplus.net
Nov 27, 2014, 8:13:41 AM11/27/14
to
all understand the same thing.
First, the final calculation of odds between players is based on
a previous calculation of something, which in turn is based on a
previous calculation of something else and so on (down to the
level of equity lost by not making the best move)...
Second, I am not asking anybody to predict anything at all but to
verify after the fact if and how accurate a prediction would have
been.
Let me illustrate step by step. It goes like this:
1- I play 100 games of 1 point against some bot, for $1/game to be
weighted/prorated by my odds of winning against it, which will be
calculated based on skill shown at the completion of 100 games.
2- As we play, bot assigns an equity to each possible move at each
turn and when either side makes an inferior move to the one it
deems the best, it says that a certain amount of equity is lost and
all equity losses for both sides are added up until the completion
of 100 games.
3- Then the total amounts of lost equity are translated into error
rates for each side. Let's say mine comes out to be 6.
4- Then based on Snowie ER = 33 ELO, for example, the ELO difference
between the sides will be calculated. With the bot's ER being zero,
let's say then that this will compute to 6*33=200.
5- Then based on the ELO difference, my probability of winning will
come out to be 44% vs the bot's 56%.
6- So, with this, in a fair bet I would put in $44 vs the bot's $56
and if it all ended as predicted by the calculations, I would win
$44 vs the bot's $56, thus both sides would break even. The purpose
of this whole thing is to prove that the bot's calculations can be
depended on enough to safely bet money on (i.e. without the risk of
losing anything if the predictions are accurate).
7- If they are not accurate, then either side would win/lose more.
For example, if I only win 30% then the bot would make $14. But if
I win 52%, then I would make $8.
8- The above can be modified for cubeful money sessions or matches
of any given length as long as the principle is the same.
I hope it is all clear to everyone now. Anybody who defers to the
bots as the strongest judge of best moves should have no problem
accepting this challenge. I would expect that especiallty the bot
developers would want to conduct this experiment to prove so much
more how strong their bots are.
9- If somebody would say that they defer to bots only 95%, 85% or
whatever of the time, we can adjust final probability of winning
accordingly (i.e. approximately accurate within 5%, 10%, etc.)
I am sure we will hear again that some people's time is worth too
much to waste on a break even bet but I can say the same also.
Let's all remember that we would be doing this for the sake of
science and backgammon.
If the incidental consequence happens that one side will win a few
dollars, well, so be it... ;)
Okay, enough said. It's time now for all bot worshippers to step
forward and bet their money.
I nobody can, then I will post the second part of this article,
suggesting what they can do instead... :))
MK
Tim Chow
Nov 27, 2014, 8:55:59 AM11/27/14
to
On Thursday, November 27, 2014 6:30:59 AM UTC-5, mu...@compuplus.net wrote:
> As usual and expected from you, once again you repeat one of
> your cube decision examples during final stages of bear off,
> because you can't do any better than that and second because
> that's basically the scope of the so-called "cube skill"...!
I picked it because it's the simplest example that will make the point I'm trying to make. Why use a more complicated example when a simple one will do?
> As usual and expected from you, once again you repeat one of
> your cube decision examples during final stages of bear off,
> because you can't do any better than that and second because
> that's basically the scope of the so-called "cube skill"...!
> The reason you can expect that is because your example above
> is only one notch above a coin toss in terms of simplicity.
> If you were to look at other positions during the early and
> middle stages of the game, the likelihood of your expectation
> coming true would diminish and you would become less and less
> capable of making such predictions. This is one of my points.
> > The rollouts are "bot-subjective" but still not worthless. They
> > tell me not to double against Neil.
>
> What you or anyone else interprets and learns from rollouts
> is irrelevant to my argument/question, which basically boils
> down to asking "Can you beat the bots making moves that are
> considered inferior by the bot?", thus question the ability
> of the bots in determining what moves are inferior/superior.
If instead by "beating the bots" you mean beating it with greater than 50% probability in a series of 11-point matches, then I don't think so.
> Not only did you pick a doubling decision but a doubling
> decision during the final rolls of a game because that's
> all you are capable of... :( Next time, try to do a little
> better.
> Do you mean the error rate would vary from person to person?
> Fine, just pick a person with a certain error rate according
> to a certain bot. Now answer these questions: what would your
> expectations be about that person's chances of beating that bot,
> how accurate would be your expectations and how accurate the
> error rate or any other numbers you use in arriving at your
> expectation? Is it really that difficult for you folks to
> understand what I am asking? If somebody understands it and
> notices that I am not asking it in a way generally understable
> to all, can he please translate it for everybody...
> > I would, however, stand behind the same "lower end of the
> > approximation" that I have stated before. Namely, I believe
> > that XGR+ would beat any human in a long series of, say,
> > 11-point matches with at least 50% probability.
>
> That's not an approximation. That's the odds in a coin toss! :))
> It's pathetic that you can't do any better to stand behind a bot
> that you worship as the strongest on the planet. :((
I'll give you a chance now to admit it. Please state clearly that you won't play XGR+ in a series of 11-point matches at even odds for real money, because you're afraid you'll lose. Well, will you admit it?
> And why not GBUBG? Why do you keep promoting eXtremeGarbage+ if
> you can't even stand behind what supposedly makes it stronger??
> Okay, so bot can improve by keeping a running total of its
> opponents error rate, etc. and begin to notice that it/he
> is weaker, and start making wrong doubling decisions like
> the one you gave above, because then its opponent would be
> likely to drop and the bot would have purposefully turned
> an inferior move into a superior move... (??)
> But, for this, first you guys have to propose that there is
> "continuity in the game" and be able to analyze beyond each
> individual position separately, as it is done now.
> "Mistakes" as defined by whom? Bots?? :) You still don't get it
> that I am questioning whether a mistake according to the bot may
> actually be the right move against that bot...!
In any case, you can settle all these questions easily by giving a public demonstration that you can beat XGR+ with greater than 50% probability in a series of 11-point matches. Oh wait, I forgot. You're a coward. Never mind.
> And I am patiently trying to pin you down on "how often?", "how
> much not always?", etc. Frankly, the only I hope I have at this
> point is that maybe somebody else reading our discussion will
> be able to answer those and have to balls to admit what may not
> add up...
> In fact, I ma not sure if you can even defend the soundness of
> the concepts and the calculations/formulas themselves.
>
> Could it be that they are simply elaborate bullshit...?? ;)
---
Tim Chow
Tim Chow
Nov 27, 2014, 8:58:09 AM11/27/14
to
On Thursday, November 27, 2014 8:13:41 AM UTC-5, mu...@compuplus.net wrote:
> The purpose
> of this whole thing is to prove that the bot's calculations can be
> depended on enough to safely bet money on (i.e. without the risk of
> losing anything if the predictions are accurate).
It's not possible to prove something that is already known to be false.
> The purpose
> of this whole thing is to prove that the bot's calculations can be
> depended on enough to safely bet money on (i.e. without the risk of
> losing anything if the predictions are accurate).
---
Tim Chow
Bradley K. Sherman
Nov 27, 2014, 10:26:34 AM11/27/14
to
<mu...@compuplus.net> wrote:
> ...
That's the experiment that we are conducting in this newsgroup.
We aspire to find someone who claims that there is a human way
to play a bot that is *better than the bot*. In fact, I, and
probably others here, would pay off a wager to such a person.
But there are lots of ways that humans can defeat even the best
bot *if* we set up certain conditions. For example, if we require
the bot to read the tournament schedule, find its table, pick up
and roll the dice and move the physical pieces, then the humans
will trounce the bot.
Gnubg keeps telling me that I am an intermediate player. That
is humbling, but is completely irrelevant to the fact that over
the long run (for short values of long) gnubg is much better
than I am. We had long arguments in the 70's about how to
rate players. There's no right way to assign a number to a
human player's strength. Consider the 65-year-old with a 2200
ELO chess rating who hasn't played competitively for twenty
years playing a 21-year-old who has jumped 300 points in the
past year, from 1900 to 2200. In most cases the 21-year-old
is going to trounce his elder despite the identical rating.
(And beware the 13-year-old with a 1500 rating!)
--bks
> ...
>I'm talking about a human playing against a bot, in a way
>different than a bot would play against itself.
> ...
>different than a bot would play against itself.
That's the experiment that we are conducting in this newsgroup.
We aspire to find someone who claims that there is a human way
to play a bot that is *better than the bot*. In fact, I, and
probably others here, would pay off a wager to such a person.
But there are lots of ways that humans can defeat even the best
bot *if* we set up certain conditions. For example, if we require
the bot to read the tournament schedule, find its table, pick up
and roll the dice and move the physical pieces, then the humans
will trounce the bot.
Gnubg keeps telling me that I am an intermediate player. That
is humbling, but is completely irrelevant to the fact that over
the long run (for short values of long) gnubg is much better
than I am. We had long arguments in the 70's about how to
rate players. There's no right way to assign a number to a
human player's strength. Consider the 65-year-old with a 2200
ELO chess rating who hasn't played competitively for twenty
years playing a 21-year-old who has jumped 300 points in the
past year, from 1900 to 2200. In most cases the 21-year-old
is going to trounce his elder despite the identical rating.
(And beware the 13-year-old with a 1500 rating!)
--bks
0 new messages