Positions that the bots screw up

[Tim Chow]

In another thread, Neil Robins said that he hasn't ever encountered a
position that the bots screw up even up on rollout. I'm quite
surprised to hear him say that because I think it's quite easy to
construct examples. Now, I don't own XG2, so my experience with it is
limited to back-and-forth exchanges with people who do own it, but I'm
pretty sure that XG2 suffers from almost all the same problems.

Here's a concrete example, the first that comes to mind. It's a
Kauder paradox position that I remember seeing in Robertie's "Advanced
Backgammon." The correct cube action is Double/Beaver. What does XG2
say when you roll out this cube decision?

XGID=-NcccccA------------------:0:0:1:00:0:0:1:0:10

X:Player 1 O:Player 2
Score is X:0 O:0. Money session, Jacoby
+13-14-15-16-17-18------19-20-21-22-23-24-+
| | | |
| | | |
| | | |
| | | |
| | | |
| |BAR| |
| | | 14 |
| | | X |
| | | O O O O O X |
| | | O O O O O X |
| X | | O O O O O X |
+12-11-10--9--8--7-------6--5--4--3--2--1-+
Pipcount X: 21 O: 315 X-O: 0-0
Cube : 1
X on roll, cube action

Analyzed in 4 ply
Player Winning Chances: 94.00% (G: 92.62% B: 31.05%)
Opponent Winning Chances: 6.00% (G: 0.05% B: 0.01%)

Cubeless Equities: No Double=+0.880, Double=+4.177

Cubeful Equities:
No Double: +1.000
Double/Take: +4.135 (+3.135)
Double/Drop: +1.000

Best Cube action: Double / Drop

eXtreme Gammon Version: 1.21

---
Tim Chow

[Grunty]

I wouldn't take a paradoxical position as an example of bot's (nor
human's!) misjudgement.

[Paul]

If "encountered" means "encountered in practical play" then Grunty has a point.
The given position is very atypical in practical terms.

However, in context, I think Neil Robins meantt that, at a theoretical level, he (Neil) didn't know of positions which the bot radically misunderstands.

A mathematician (like Tim) typically treats that as a find-a-counterexample problem. Here, the counterexample need only be legal, not practical.

Tim has most gloriously solved the above counterexample problem, and this stellar achievement should not be denigrated.

Another problem might be: Find a position that appears practically plausible among strong players and that XG2 misevaluates by rollout by more than 0.5 points.

Paul Epstein

[Walt]

On 9/5/2012 9:45 AM, Tim Chow wrote:
> The correct cube action is Double/Beaver. What does XG2
> say when you roll out this cube decision?

How do you know the "correct" cube action? XG2 rollout says it's a
monster cash. Is this one of those "can't figure out how to roll the
prime forward" positions?

BTW, I couldn't figure out how to turn on Beavers in XG, so I don't
think this rollout accounts for Beavers.

XGID=-NcccccA------------------:0:0:1:00:0:0:1:0:10
X:Player 1 O:Player 2

Score is X:0 O:0. Unlimited Game, Jacoby

+13-14-15-16-17-18------19-20-21-22-23-24-+
| | | |
| | | |
| | | |
| | | |
| | | |
| |BAR| |
| | | 14 |
| | | X |
| | | O O O O O X |
| | | O O O O O X |
| X | | O O O O O X |
+12-11-10--9--8--7-------6--5--4--3--2--1-+

Pip count X: 21 O: 315 X-O: 0-0

Cube: 1
X on roll, cube action

Analyzed in Rollout
No double
Player Winning Chances: 78.57% (G:76.81% B:29.45%)
Opponent Winning Chances: 21.43% (G:5.16% B:0.50%)
Double/Take
Player Winning Chances: 86.88% (G:84.60% B:28.78%)
Opponent Winning Chances: 13.12% (G:3.89% B:0.21%)

Cubeless Equities: No Double=+1.577, Double=+3.661

Cubeful Equities:
No double: +0.698 (-0.302)
Double/Take: +3.542 (+2.542)
Double/Pass: +1.000

Best Cube action: Double / Pass

Rollout:
1296 Games rolled with Variance Reduction.
Moves: 3-ply, cube decisions: XG Roller
Confidence No Double: � 0.067 (+0.631..+0.765)
Confidence Double: � 0.232 (+3.310..+3.774)

Double Decision confidence: 100.0%
Take Decision confidence: 100.0%

Duration: 1 hour 16 minutes

eXtreme Gammon Version: 2.03



--
//Walt

[peps...@gmail.com]

If you make the following assumptions, you can easily show (by simple equity calculations) that the correct cube action is double-beaver.

Assumption 1) If X rolls a 6, X always gets a gammon.
Assumption 2) If X misses a 6, O has a cash.

Assumption 1) can also be "proven" mathematically as follows. Any 6 leaves X 14 crossovers from winning leaving X with a 7-roll position in X's worst-case scenario. However O is 46 crossovers from saving the gammon -- more than 11 rolls away.

It needs only to demonstrate assumption 2 which can't be proved mathematically, and is a matter of experience.

Yes, it's about the bot's inability to judge how easy it is to roll a prime forward.

However, there's a catch at the end.

The problem with the above analysis (I think Tim changed Robertie's position) is that X is very likely to score a backgammon.

If we assume that X always gets a backgammon with a 6 (not very far at all from the truth), then after a take, X wins 11/36 * 6 and loses only 25/36 * 2.
Definitely not a beaver -- Tim must be wrong.

But it can't be a pass either so XG2 messed up too (and that is after all what the NeilTim discourse is about). It can't be a pass because even if we help X by making all X's 6's backgammons, O still has an easy take.

But to accept that analysis, you have to accept that O cashes when X misses a 6. But surely that's true. Isn't it?

Paul Epstein

[Grunty]

> If "encountered" means "encountered in practical play" then Grunty has a point.
> The given position is very atypical in practical terms.
>
> However, in context, I think Neil Robins meantt that,
> at a theoretical level, he (Neil) didn't know of positions
> which the bot radically misunderstands.

Talking of context. In thread "Requesting proof", Aug 13, the
subthread Tim alluded went as follows:

checkmug...@yahoo.com wrote:
Even eXtreme Gammon version 2 gets confused with certain superbackgame
positions, regardless of setting (including full rollout). It's not
hard to set up a prop where you can beat the living daylights out of
XGR+.

then Frank $tudt wrote:
A few reference positions would be nice.

then Neil Robins wrote:
I too would like to see these superbackgame positions that eXtreme
Gammon
version 2 gets confused with regardless of setting (including full
rollout).
I don't believe I've ever seen even one.

-- end of transcript --

All this conversation was about "practically plausible" positions
(e.g.: superbackgames) rather than a purely theoretical question.
Therefore, if you're to follow up on this matter, you should present
practically plausible positions, instead of paradoxical ones.

> A mathematician (like Tim) typically treats that as a
> find-a-counterexample problem.  Here, the counterexample
> need only be legal, not practical.
>
> Tim has most gloriously solved the above counterexample
> problem, and this stellar achievement should not be denigrated.
>
> Another problem might be:  Find a position that appears
> practically plausible among strong players and that XG2
> misevaluates by rollout by more than 0.5 points.

That was actually Neil's request.

[peps...@gmail.com]

Grunty,

I agree with you now, but you have actually underestimated the gravity of Tim's felony.

I don't believe Tim's verdict of double-beaver can possibly be correct, as I argued earlier.

The original position is not a Kauder paradox position.

So not only is Tim's example irrelevant to Neil's request, but it is also wrong.

Paul

[Tim Chow]

First of all, Paul is right that I screwed up Robertie's position. I
was working from memory, since I don't own Robertie's "Advanced
Backgammon." However, I think the fix is relatively easy; a small
adjustment, such as shifting the position over a pip, should drop the
backgammon rate down to where it's a double/beaver. Perhaps someone
who owns Robertie's book can post the corrected position?

On Sep 5, 7:02 pm, Grunty <gruntingdw...@yahoo.com> wrote:
> All this conversation was about "practically plausible" positions
> (e.g.: superbackgames) rather than a purely theoretical question.
> Therefore, if you're to follow up on this matter, you should present
> practically plausible positions, instead of paradoxical ones.

Well, I was the one who started the discussion, and I didn't say
anything about "practically plausible." I did use the term
"superbackgame," but does that term imply that the position is
practically plausible? That's news to me. I also talked about
setting up a prop. When setting up a prop, the understanding is that
you're free to set up an unusual position.

Also, I think you may be getting confused by the use of the term
"paradox" here. Conventionally, a position where the technically
correct cube action is double/beaver is called a "Kauder paradox"
because it violates our intuition that there could possibly be a
position like that. It doesn't mean that the position isn't likely to
arise in practice. Robertie's position happens to be practically
implausible, but there are other Kauder paradox positions that can
arise, and have arisen, in practice. Similarly, the "Jacoby paradox"
refers to a particular situation that you might not think was possible
but that arises in practice quite frequently. Anyway, my point is
that whether a position is a "Kauder paradox" is a completely
orthogonal question to whether it is practically plausible.

Getting back to the original question, the point I was making (and
I'll point out again that I was the one who was speaking first, not
Frank or Neil) was that it's quite possible to set up positions that
confuse the bot. The first position that came to mind was Robertie's
position, and I believe that that proves my point, assuming that XG2
doesn't analyze it correctly upon rollout.

Now, somewhere along the line, the words "practically plausible" got
inserted. I don't see it in what Frank or Neil wrote, but let me
assume, as Grunty seems to suggest, that part of the definition of the
word "superbackgame" is that the position be "practically plausible."
I'm not sure exactly what counts as practically plausible. I agree
that Robertie's position is implausible. However, in my playing
group, when we're playing for fun, we sometimes get into positions
where all fifteen checkers get sent back. Set up a six-prime plus
three spares somewhere near the opponent's home board, holding back a
single opposing checker. Is that "practically plausible"? Such
positions have arisen in real games that I've played. I'm pretty sure
XG2 would screw it up if you rolled it out.

I have another position which I will try to remember to post tomorrow
that was less extreme than that and even more "practically
plausible." The trouble with such examples is that it becomes harder
to prove that the bot is really screwing up. How do you know?
Robertie's position has the advantage that you can analyze it by hand
and get the right answer with high confidence. If I set up a more
complicated but more plausible position and claim that the bot rollout
is wrong, how can I justify the claim? Well, one thing one can do is
to evaluate the position at different ply levels and see what
happens. If the evaluations fluctuate wildly as you change the ply
level, then I consider that a sign that the bot doesn't know what's
going on. Now you could of course object, and say that even though
the bot changes its mind wildly from 3-ply to 4-ply to XGR, etc., the
all-powerful *rollout* must be telling us the gospel truth. O.K. I
can't prove you're wrong, but it doesn't seem plausible to me.
Anyway, that's another reason for choosing an artificial position
that's easy to analyze, so that these kinds of arguments don't get in
the way.

---
Tim Chow

[Tim Chow]

On Sep 5, 7:02 pm, Grunty <gruntingdw...@yahoo.com> wrote:

> checkmug...@yahoo.com wrote:
>
> Even eXtreme Gammon version 2 gets confused with certain superbackgame
> positions, regardless of setting (including full rollout). It's not
> hard to set up a prop where you can beat the living daylights out of
> XGR+.

I just noticed something. It looks like you're attributing my words
to Stick. Perhaps that's part of the reason for the confusion.

---
Tim Chow

[peps...@gmail.com]

The phrase "practically plausible" originated with myself. I interpreted Grunty's complaint as being that your position here is not practically plausible (and Grunty didn't disagree with my interpretation).

Robertie's approach to the position (if I remember rightly) is to leave X's 14 checkers on the ace but to shift the rest of the position back.
To give a really clean demonstration of the paradox, we want X's backgammon rate with a 6 to be 0% and X's gammon rate with a 6 to be 100%. However, I don't think this is possible with a small adjustment of the original position (but I'm not sure).
There are two approaches to the backward shift. One is to preserve the 100% (gammons and backgammons) but to tolerate a small backgammon rate. The other is to make the backgammons for X exactly 0 but to tolerate a small percentage of missed gammons.
The latter is Robertie's approach but Robertie didn't mention (or didn't realise) that X's 6s don't lead to gammons with 100% certainty.
The double-beaver action, under the theoretically simplified assumptions, is clear enough that the example surely works if X's 6s lead to gammons with a probability of 99% or greater.
I'm not sure whether Robertie did this but I would try a fix by leaving X's 14 checkers on the ace point and shifting the rest of the position 3 pips back. That seems to leave 0 backgammons for X and easily enough gammons for X.

It's not completely obvious to me that O can cash if X misses a 6, however. (I'll trust the experts here, though). It seems to be asking quite a lot for O to maintain the prime so consistently over such a length of time. That sort of judgment is best tested by experience, though. I think the cumulative experience is that rolling forward a prime, even over considerable distances, is easier than it looks.

Paul Epstein

[Walt]

On 9/5/2012 9:45 AM, Tim Chow wrote:

The correct cube action is Double/Beaver. What does XG2
> say when you roll out this cube decision?
>

I figured out how to turn on beavers in XG. Here's the rollout. It's
barely different than the non-beaver rollout.

I don't think it's correct. Seems to me that O has a cash when X fails
to roll a 6, which happens 70% of the time.



XGID=-NcccccA------------------:0:0:1:00:0:0:3:0:10

X:Player 1 O:Player 2

Score is X:0 O:0. Unlimited Game, Jacoby Beaver

+13-14-15-16-17-18------19-20-21-22-23-24-+
| | | |
| | | |
| | | |
| | | |
| | | |
| |BAR| |
| | | 14 |
| | | X |
| | | O O O O O X |
| | | O O O O O X |
| X | | O O O O O X |
+12-11-10--9--8--7-------6--5--4--3--2--1-+

Pip count X: 21 O: 315 X-O: 0-0

Cube: 1
X on roll, cube action

Analyzed in Rollout
No double

Player Winning Chances: 78.42% (G:76.79% B:29.39%)
Opponent Winning Chances: 21.58% (G:4.96% B:0.40%)
Double/Take
Player Winning Chances: 86.85% (G:84.58% B:28.79%)
Opponent Winning Chances: 13.15% (G:3.96% B:0.20%)

Cubeless Equities: No Double=+1.577, Double=+3.658

Cubeful Equities:
No double: +0.679 (-0.321)
Double/Take: +3.538 (+2.538)

Double/Pass: +1.000

Best Cube action: Double / Pass

Rollout:
1296 Games rolled with Variance Reduction.
Moves: 3-ply, cube decisions: XG Roller

Confidence No Double: � 0.068 (+0.611..+0.747)
Confidence Double: � 0.232 (+3.307..+3.770)

Double Decision confidence: 100.0%
Take Decision confidence: 100.0%

Duration: 43 minutes 16 seconds

[Tim Chow]

On Sep 5, 9:36 pm, I wrote:
> I have another position which I will try to remember to post tomorrow
> that was less extreme than that and even more "practically plausible."

The position is shown below. I exhibit it as a doubling decision not
because I necessarily think that doubling is plausible, but to display
XG's cubeless estimates. The analysis is courtesy of a friend of mine
who owns XG2. Here are his comments:

>XG 2.03 is still quite confused by it. For instance, if I put X's checker on the
>board somewhere, say on O's 6-point, then a 3-ply analysis of O's cube action says
>that X has a 91.76% chance of winning, with all wins being gammons and 74.04% of
>games ending in a backgammon. A 4-ply analysis says that X's winning chances are
>only 53.04%, with 20.62% of games being gammons or better and 14% being backgammons.
>Under the circumstances, I don't think any rollout results are meaningful.

Reading his comments again, I'm not sure exactly what he means by
"putting X's checker on the board somewhere"; I think he means moving
X's blot around on the board. In any case, the basic point is clear.
If anyone wants to roll this out and argue that the rollout results
are reliable, I'm all ears.

Is this position practically plausible? It arose in an actual game
between friends of mine, but maybe the fact that the position actually
arose in practice isn't enough to ensure that it's "practically
plausible."

XGID=---a----B-----BBBBB-Abd-B-:1:1:1:00:0:0:3:0:10

X:Player 1 O:Player 2

Score is X:0 O:0. Money session, Jacoby Beaver
+13-14-15-16-17-18------19-20-21-22-23-24-+
| X X X X X | | X O O X |
| X X X X X | | O O X |
| | | O |
| | | O |
| | | |
| |BAR| |
| | | |
| | | |
| | | | +---+
| X | | | | 2 |
| X | | O | +---+
+12-11-10--9--8--7-------6--5--4--3--2--1-+
Pipcount X: 244 O: 42 X-O: 0-0
Cube : 2, X own cube

X on roll, cube action

Analyzed in XG Roller+
Player Winning Chances: 18.40% (G: 0.00% B: 0.00%)
Opponent Winning Chances: 81.60% (G: 77.47% B: 29.26%)

Cubeless Equities: No Redouble=-1.699, Redouble=-3.842

Cubeful Equities:
No Redouble: -2.134
Redouble/Take: -3.951 (-1.817)
Redouble/Drop: +1.000 (+3.134)

Best Cube action: No Redouble / Take

---
Tim Chow

[Grunty]

Thanks for the rollout.

Now would you be so kind to roll out the following position, which
mirrors (X moves counter-clockwise) the one posted in "Advanced
Backgammon", vol.I "Positional Play", last problem in chapter "Action
Doubles"; as an example of the Kauder Paradox.

+13-14-15-16-17-18------19-20-21-22-23-24-+
| | | |
| | | |
| | | |
| | | |
| | | |
| |BAR| |
| | | 6 6 |
| | | X X |
| O O O | | O O X X |
| O O O | | O O X X X |
| X O O O | | O O X X X |
+12-11-10--9--8--7-------6--5--4--3--2--1-+

Robertie starts out saying: "A very implausible position to be sure,
but one that fits."

[Walt]

On 9/6/2012 2:36 PM, Grunty wrote:
>
> Now would you be so kind to roll out the following position, which
> mirrors (X moves counter-clockwise) the one posted in "Advanced
> Backgammon", vol.I "Positional Play", last problem in chapter "Action
> Doubles"; as an example of the Kauder Paradox.
>

Here ya go.

I'm not seeing 78% wins for X here. Just like the other position, it
seems that O has a cash whenever X misses throwing a 6.

BTW, I never play for money, and almost always play match play, so I'm
not very well versed in Jacoby & Beavers. But my understanding is that
the Kauder paradox is an artifact of needing to turn the cube to
activate gammons. The fact that Jacoby rule gives rise to a paradox
makes me think it's not a good rule. (Not that anybody is going to
change anything on my opinion.)


XGID=-FFB-cccccA---------------:0:0:1:00:0:0:3:0:10

X:Player 1 O:Player 2

Score is X:0 O:0. Unlimited Game, Jacoby Beaver
+13-14-15-16-17-18------19-20-21-22-23-24-+
| | | |
| | | |
| | | |
| | | |
| | | |
| |BAR| |

| | | 6 6 |
| | | X X |
| O O O | | O O X X |
| O O O | | O O X X X |
| X O O O | | O O X X X |
+12-11-10--9--8--7-------6--5--4--3--2--1-+

Pip count X: 34 O: 270 X-O: 0-0

Cube: 1
X on roll, cube action

Analyzed in Rollout
No double

Player Winning Chances: 78.35% (G:77.17% B:0.43%)
Opponent Winning Chances: 21.65% (G:1.40% B:0.13%)
Double/Take
Player Winning Chances: 62.09% (G:58.44% B:0.08%)
Opponent Winning Chances: 37.91% (G:7.68% B:0.49%)

Cubeless Equities: No Double=+1.328, Double=+1.490

Cubeful Equities:
No double: +0.938 (-0.062)
Double/Take: +1.238 (+0.238)

Double/Pass: +1.000

Best Cube action: Double / Pass

Rollout:
1296 Games rolled with Variance Reduction.
Moves: 3-ply, cube decisions: XG Roller

Confidence No Double: � 0.034 (+0.904..+0.973)
Confidence Double: � 0.119 (+1.119..+1.357)

Double Decision confidence: 100.0%
Take Decision confidence: 100.0%

Duration: 47 minutes 36 seconds

[Paul]

On Friday, September 7, 2012 3:29:52 PM UTC+1, Walt wrote:
...

>
> Here ya go.
>
>
>
> I'm not seeing 78% wins for X here. Just like the other position, it
>
> seems that O has a cash whenever X misses throwing a 6.
>
>
>
> BTW, I never play for money, and almost always play match play, so I'm
>
> not very well versed in Jacoby & Beavers. But my understanding is that
>
> the Kauder paradox is an artifact of needing to turn the cube to
>
> activate gammons. The fact that Jacoby rule gives rise to a paradox
>
> makes me think it's not a good rule. (Not that anybody is going to
>
> change anything on my opinion.)
>

Your understanding of the Kauder paradox seems exemplary to me. Even Walter Trice could not have bettered it.

The problem, as I see it, is not that the Jacoby rule gives rise to a paradox, but rather the specific nature of the Kauder paradox. I don't see a problem with a rule giving rise to paradoxes in general. For example, every person's life has paradoxical elements, but I wouldn't argue that therefore deciding to have a child and thereby cause paradoxes is necessarily wrong.

Jacoby gives rise to (at least) two paradoxes, Jacoby and Kauder. I don't see any problem at all with the Jacoby paradox. In fact, it makes the game more interesting. There is a big problem with the Kauder paradox, though. The whole point of the beaver concept is that it's a tool for the cubed player to punish a weak double. It is therefore highly problematic to see beavers arise from optimal play. However, the fraction of beavers that are Kauder-paradox-beavers must be tiny.

The main problem with the Jacoby rule (in my opinion) is that it simplifies the game, and thereby increases the luck element. The simplification is that the favourite never has to consider whether the position is too-good if the cube is centred.

It doesn't seem at all certain that the Jacoby rule will remain standard for money games. Firstly, the cube is a fairly recent invention so changing Jacoby isn't like changing rules which have a long and cherished tradition. Secondly, quite a few backgammon celebrities (Giants) are against Jacoby but I don't know whether they use it in their own games. Thirdly, there is at least one prominent bg site that doesn't use Jacoby (or didn't last time I checked).

Stick, if you're listening, what is your Jacoby policy? Do you like to use it in your money games?

Thank you,

Paul Epstein

[Walt]

On 9/7/2012 10:29 AM, Walt wrote:

> I'm not seeing 78% wins for X here. Just like the other position, it
> seems that O has a cash whenever X misses throwing a 6.

Here's a rollout of o"s cube decision when X fails to roll a 6. I
stoped it after 15 minutes because I'm not expecting it to get any better.

No Redouble/Beaver? WTF?



XGID=-FFB-cccccA---------------:2:-1:-1:00:0:0:3:0:10

X:Player 1 O:Player 2

Score is X:0 O:0. Unlimited Game, Jacoby Beaver

+12-11-10--9--8--7-------6--5--4--3--2--1-+
| | | | +---+
| | | | | 4 |
| | | | +---+

| | | |
| | | |
| |BAR| |
| | | 6 6 |
| | | X X |
| O O O | | O O X X |
| O O O | | O O X X X |
| X O O O | | O O X X X |

+13-14-15-16-17-18------19-20-21-22-23-24-+

Pip count X: 34 O: 270 X-O: 0-0

Cube: 4, O own cube
O on roll, cube action

Analyzed in Rollout
No redouble
Player Winning Chances: 53.97% (G:7.53% B:0.00%)
Opponent Winning Chances: 46.03% (G:39.65% B:0.04%)
Redouble/Take
Player Winning Chances: 35.41% (G:5.98% B:0.00%)
Opponent Winning Chances: 64.59% (G:55.09% B:0.24%)

Cubeless Equities: No Double=-0.242, Double=-1.570

Cubeful Equities:
No redouble: -0.081
Redouble/Beaver: -2.000 (-1.919)
Redouble/Pass: +1.000 (+1.081)

Best Cube action: No redouble / Beaver
Percentage of wrong pass needed to make the double decision right: 39.0%

Rollout:
206 Games rolled with Variance Reduction.

Moves: 3-ply, cube decisions: XG Roller

Confidence No Double: � 0.167 (-0.248..+0.086)
Confidence Double: � 0.794 (-2.794..-1.206)

Double Decision confidence: 100.0%
Beaver Decision confidence: 100.0%

Duration: 16 minutes 12 seconds

[Neil Robins]

"Tim Chow" <tchow...@yahoo.com> wrote in message
news:b965748d-963f-48c0...@fm12g2000vbb.googlegroups.com...

So what if you can set up a position that cofuses the bot. I look at
practical positions from real games. You might get into games where all
fifteen checkers get sent back but I think it was about thirty years ago
that I last did. Furthermore, I don't look at any nonsense invoving the
Jacoby rule, which I never enable.

BTW others have pointed out what I actually said in another thread. How you
put that at the start of this thread seems more like deliberate
misrepresentation than poor paraphrasing to me.

[Walt]

On 9/7/2012 1:54 PM, Walt wrote:
> On 9/7/2012 10:29 AM, Walt wrote:
> > XGID=-FFB-cccccA---------------:2:-1:-1:00:0:0:3:0:10
> X:Player 1 O:Player 2
>
> Score is X:0 O:0. Unlimited Game, Jacoby Beaver
> +12-11-10--9--8--7-------6--5--4--3--2--1-+
> | | | | +---+
> | | | | | 4 |
> | | | | +---+
> | | | |
> | | | |
> | |BAR| |
> | | | 6 6 |
> | | | X X |
> | O O O | | O O X X |
> | O O O | | O O X X X |
> | X O O O | | O O X X X |
> +13-14-15-16-17-18------19-20-21-22-23-24-+
> Pip count X: 34 O: 270 X-O: 0-0
> Cube: 4, O own cube
> O on roll, cube action

Out of curiosity, I played this position out, assuming X doesn't roll a
6. I played about 40 games against XG2 on "Professional" level, with a
roughly equal number of trials playing X and O.

My observations:

Rolling the prime forward all the way around the board is not as easy as
I had assumed. 44 & 55 are usually very awkward as is 65 in many
positions. It was not always possible to keep a solid 6-prime and once
the solid 6-prime is broken, it's hard to re-establish it, especially
after 44 or 55. I often had to allow fly shots to leap the prime and
since many such opportunities to leap come up, maybe X has enough to
take this position.

Playing as O, I won about 2 out of 3 against XG, but about half of XG's
wins were gammons. XG seemed to make very reasonable plays as X -
perhaps because most rolls were forced. I probably made some blunders,
but I can't trust XG to point them out to me.

Playing as X, I also won about 2 out of 3, with about half of those
gammons. I see what people say about XG not understanding how to roll
the prime forward; it refused to make anything resembling a prime on my
side of the board. Once it brought its checkers around it would form a
prime on its side of the board, but it would also unnecessarily break
that prime for no good reason that I could see.

I didn't pay much attention to the cube - I just took whatever cubes
were offered and didn't double as X. So think of these results as a
cubeless evaluation.


Bottom line is that I'm convinced that XG doesn't understand this
position. Neither do I, but the results seem to indicate that I have a
better grasp of it.

I'm now skeptical that O has a cash after X fans. XG beavers as X, and
tht's clearly wrong, but it might be a take. Robertie has been wrong
before.

--
//Walt

[Paul]

Walt,

Robertie has certainly "been wrong before." I believe that there are huge numbers of mistakes in his positions.

Your scepticism about the position is quite a close match to what I said before in this thread:

"It's not completely obvious to me that O can cash if X misses a 6, however. (I'll trust the experts here, though). It seems to be asking quite a lot for O to maintain the prime so consistently over such a length of time. That sort of judgment is best tested by experience, though. I think the cumulative experience is that rolling forward a prime, even over considerable distances, is easier than it looks."

I believe that when you say something that either agrees or disagrees with something someone has already said, you should indicate that by saying something like "I agree with Paul that ..." Otherwise, I feel that I'm not being listened to.

I mentioned that I "trust the experts" but this trust might well be misplaced, particularly in the light of your experiment. I think that if Robertie's position is discovered to be seriously wrong because O's redouble is a take, that would be quite an exciting find.

40 games is a meaningful sample size. I see (at least) three possibilities here:
1) Your results were statistically unusual. It's in fact a clear drop (when X misses the 6) but your 40 games were a very atypical sample.

2) You are correct to be sceptical, Robertie's wrong and the redouble is a take.

3) In terms of the sequences of rolls, the 40 positions were representative. However, they contain a lot of blunders which distort the findings. In particular, you made a lot of blunders when trying to roll the prime forward for O.

Of course, scenarios 1 to 3 are not mutually exclusive.

Roughly speaking, I would say that my three scenarios are all possible. I believe scenario 3 but scenario 2 seems very plausible too.

Paul's intuitive probabilities: Scenario 3: 75% probability, scenario 2, 20% probability, scenario 1, 5% probability. (Since the scenarios are not mutually exclusive, imagine the word "mostly" inserted in the above).

To investigate further, I think it would be a good idea to post your own play when you tried to roll the prime forward, and hope that a strong player (I'm not strong) is able to comment on whether you played correctly. Clearly, since this whole thread is motivated by a discussion of bot weaknesses, you can't rely on the bots to assess your play. If I were in your position, I would probably pay a strong player to analyse my play but not everyone enjoys getting lessons in the way that I do.

Paul Epstein

[Bradley K. Sherman]

Paul <peps...@gmail.com> wrote:
> ...

>To investigate further, I think it would be a good idea to post your own
>play when you tried to roll the prime forward, and hope that a strong

> ...

I thought the master consensus was that the bots didn't understand
the cube in this position. Are we now claiming that the bots
don't know how to roll a prime around the board? Isn't it enough
just to turn it into a cubeless game and ask XG/gnubg to play
it out a thousand times?

--bks

[Walt]

On 9/11/2012 10:06 AM, Bradley K. Sherman wrote:
> Paul <peps...@gmail.com> wrote:
>> ...
>> To investigate further, I think it would be a good idea to post your own
>> play when you tried to roll the prime forward, and hope that a strong
>> ...
>
> I thought the master consensus was that the bots didn't understand
> the cube in this position. Are we now claiming that the bots
> don't know how to roll a prime around the board?

Yes, that's been claimed many times and it's exactly what I found when I
tried it myself. You have gnubg, right? Try popping that position into
gnubg and playing as X. You'll be appalled at how poorly gnu rolls the
prime forward.

>Isn't it enough
> just to turn it into a cubeless game and ask XG/gnubg to play
> it out a thousand times?

Yes, that can be done. Unfortunately, neither bot can roll the prime
forward so the cubeless result won't reflect any real-world winning
percentage.

--
//Walt

[Bradley K. Sherman]

Walt <walt_...@yahoo.com> wrote:
>On 9/11/2012 10:06 AM, Bradley K. Sherman wrote:
>> Paul <peps...@gmail.com> wrote:
>>> ...
>>> To investigate further, I think it would be a good idea to post your own
>>> play when you tried to roll the prime forward, and hope that a strong
>>> ...
>>
>> I thought the master consensus was that the bots didn't understand
>> the cube in this position. Are we now claiming that the bots
>> don't know how to roll a prime around the board?
>
>
>Yes, that's been claimed many times and it's exactly what I found when I
>tried it myself. You have gnubg, right? Try popping that position into
>gnubg and playing as X. You'll be appalled at how poorly gnu rolls the
>prime forward.

Not necessary, I believe you. I also believe you when you say
that it's not as easy as some claim.

--bks

[Tim Chow]

On Sep 8, 6:28 pm, "Neil Robins" <neil.robi...@ntlworld.com> wrote:
> So what if you can set up a position that cofuses the bot. I look at
> practical positions from real games. You might get into games where all
> fifteen checkers get sent back but I think it was about thirty years ago
> that I last did. Furthermore, I don't look at any nonsense invoving the
> Jacoby rule, which I never enable.
>
> BTW others have pointed out what I actually said in another thread. How you
> put that at the start of this thread seems more like deliberate
> misrepresentation than poor paraphrasing to me.

Where is all this coming from?

When I said I 'started the discussion,' I was referring to the portion
of the discussion that Grunty quoted. You didn't say anything in that
thread until after I had said, "Even eXtreme Gammon version 2 gets

confused with certain superbackgame positions, regardless of setting
(including full rollout). It's not hard to set up a prop where you

can beat the living daylights out of XGR+." Then Frank said, "A few
reference positions would be nice." Only then did you ask for
superbackgame positions that the bot gets confused about even upon
full rollout. The fact that you used the terms "superbackgame" and
"including full rollout," which were terms that *I* introduced into
the discussion, sure makes it sound like you wanted me to give an
example of what *I* was talking about. I used the word "prop," and
Frank asked for "reference positions." Neither of these terms implies
that the position in question has to be one that arises frequently in
practice.

I explicitly said in the other thread that I wasn't sure if one could
trick the bot into getting into these positions. So it should have
been clear to anyone paying attention that I wasn't claiming anything
about positions that arise frequently in practice, even if you're
intentionally angling for them.

Now that I give an example of what I'm talking about, you're accusing
me of deliberate misrepresentation? That's extremely annoying. I
don't mind that I haven't given you what you were really looking for,
since only now have you clarified that, but for you to turn around and
accuse me of deliberate misrepresentation is sinking close to Murat's
level.

---
Tim Chow

[Tim Chow]

On Sep 11, 9:15 am, Paul <pepste...@gmail.com> wrote:
> I mentioned that I "trust the experts" but this trust might well
> be misplaced, particularly in the light of your experiment.  I
> think that if Robertie's position is discovered to be seriously
> wrong because O's redouble is a take, that would be quite an
> exciting find.

I agree. I raised this question on BGOnline some time ago. Here's
the opinion of one more expert (Casper van der Tak) that it's a drop.

http://www.bgonline.org/forums/webbbs_config.pl?read=69845

I think Daniel Murphy would agree, though I can't quite find an exact
quote. Here's a post where he gives some information about beating
GNU 2-ply using the super-backgame strategy.

http://www.bgonline.org/forums/webbbs_config.pl?read=51338

In another post, Bradley asked:

> Are we now claiming that the bots don't know how to roll a

> prime around the board? Isn't it enough just to turn it into

> cubeless game and ask XG/gnubg to play it out a thousand times?

If you're asking whether it's possible to train a neural net to roll a
prime, the impression I've gathered from the bot programmers is that
it is possible, but that this kind of training tends to screw up its
performance in "normal" positions. Nobody seems to have quite figured
out how to solve this problem using only neural-net methods. In
particular, none of the current neural-net programs are trained in
this way.

I mentioned in one of the above BGOnline threads that rolling a prime
ought to be solvable using dynamic programming. This would do better
than a neural net ever could. However, it would be of use only in
extremely limited scenarios that virtually never arise in practice, so
it would be an academic exercise only.

---
Tim Chow

[Tim Chow]

On Sep 7, 11:38 am, Paul <pepste...@gmail.com> wrote:
> It doesn't seem at all certain that the Jacoby rule
> will remain standard for money games.

I think it's likely that it will stick around as long as chouettes
remain popular.

To predict the future of the Jacoby rule, I think you have to
understand the motivation for it, which I explained here:

http://www.bgonline.org/forums/webbbs_config.pl?read=105487

In particular, the oft-cited reason that the Jacoby rule speeds things
up should be parsed as, "the Jacoby rule causes the chouette to
*appear* to move along more rapidly." As long as this appearance is
maintained, the Jacoby rule will have strong supporters.

---
Tim Chow

[Tim Chow]

I re-read Neil Robins's last post. I see now more precisely what his
claim of deliberate misrepresentation is. He claims that:

>In another thread, Neil Robins said that he hasn't ever encountered a
>position that the bots screw up even up on rollout.

is a deliberate misrepresentation of his comment that:

>I too would like to see these superbackgame positions that eXtreme
>Gammon version 2 gets confused with regardless of setting (including
>full rollout). I don't believe I've ever seen even one.

which he made subsequent to my earlier comment in the thread that:

>Even eXtreme Gammon version 2 gets confused with certain superbackgame
>positions, regardless of setting (including full rollout). It's not
>hard to set up a prop where you can beat the living daylights out of
>XGR+.

Now that I understand more precisely what his claim of deliberate
misrepresentation is, I'm even more annoyed than I was before when I
wasn't quite as clear exactly what his accusation was.

Frankly, re-reading the entire "Requesting proof" thread again, I
still can't see where Grunty got his interpretation of "practically
plausible positions." Murat started by "requesting proof" of my
comment (in a still earlier thread) that:

> Set up an extreme backgame where you have to roll your prime all the
> way around the board and the bot will be extremely confused.

How people then managed to convince themselves that `All this
conversation was about "practically plausible" positions,' and that I
am guilty of `deliberate misrepresentation' when I interpret Neil's
comment as a request to justify my claim that I could set up a bot-
defying extreme backgame where you have to roll your prime all the way
around the board, is beyond me.

---
Tim Chow

[Walt]

On 9/11/2012 9:15 AM, Paul wrote:

> I believe that when you say something that either agrees or disagrees with something someone has already said, you should indicate that by saying something like "I agree with Paul that ..." Otherwise, I feel that I'm not being listened to.

Sorry, I wasn't paying attention. What were you saying again?



--
//Walt

[Paul]

One of the things I'm saying is that there's a strong possibility that your roll-the-prime-forward play was weak. That appears to me to be the most likely explanation of why the cash doesn't appear clear if X misses the 6.
Why not post some of your difficult prime-rolling plays here?

Paul Epstein

[Tim Chow]

On Sep 12, 8:20 am, mu...@compuplus.net wrote:

> On Tuesday, September 11, 2012 9:34:33 AM UTC-6, Tim Chow wrote:
> > Then Frank said, "A few reference positions would be nice."
> > Only then did you ask for superbackgame positions that the
> > bot gets confused about even upon full rollout.
>

> Not true! He had posted in response to my article stating
> that instead of set-up positions, I would prefer to see
> actual games leading to such positions (and not just one
> or two, but a good number of such examples)...

You are correct that Neil posted in response to your article where you
said that. However, Neil didn't say that he too would like to see
*full games* (as you did). He said, "I too would like to see these
superbackgame *positions* [emphasis mine] that eXtreme Gammon version
2 gets confused with regardless of setting (including full rollout)."
The terms "superbackgame" and "including full rollout" were terms that
I used, not you. Given that you made a clear distinction between set-
up positions and games, I can't see how anyone could be expected to
interpret Neil's request as anything other than a request for me to
exhibit explicitly the set-up positions that I was claiming existed.

If Neil had said "games" then I would completely agree with you that
he was looking for the same thing that you were looking for (games,
not set-up positions).

By the way, in that thread, you correctly read me as saying that I was
setting up positions by hand. Given that even you found it clear what
I was saying, I'm totally baffled about where Grunty got his
"practically plausible" idea from.

> Anybody who has followed the subject in several different
> threads already know what words you used, what you said
> and what you meant.

Apparently not. I'm still completely baffled as to how this
discussion has taken this turn.

---
Tim Chow

[Neil Robins]

I agree completely that GNU has no idea how to roll a prime forward,. I also
agree that XG2 has little idea of how to bring one home from the opponent's
side of the board. I don't believe I ever said anything that contradicts
that, and if you interpreted it that way, that was not my intention. I still
await a genuuine superbackgame position from a match where the move favoured
by an XG2 rollout is clearly wrong.
And please stop substitutting XG2 with "the bots".

[Walt]

On 9/12/2012 1:08 PM, Paul wrote:
>
> One of the things I'm saying is that there's a strong possibility that your roll-the-prime-forward play was weak. That appears to me to be the most likely explanation of why the cash doesn't appear clear if X misses the 6.

Possibility? It's a near certainty. I was playing the rolls quickly as
I worked through my second and third glass of Bordeaux. No maybe about it.

> Why not post some of your difficult prime-rolling plays here?

I didn't bother to save any of the games.

//Walt

[Tim Chow]

On Sep 13, 7:49 pm, "Neil Robins" <neil.robi...@ntlworld.com> wrote:
> I still await a genuuine superbackgame position from a match where
> the move favoured by an XG2 rollout is clearly wrong.

Fair enough. I see you're still not apologizing for falsely accusing
me of deliberate misrepresentation, but I suppose that's too much to
hope for.

Before I go off on a wild goose chase, maybe you can clarify what a
"genuine superbackgame position from a match" is? What makes it
"genuine"? I already posted a position that came up in a real game
between friends of mine. I'm not sure what your objection to it is?
Perhaps:

1. It was a money game, not a match.

2. My friends are weak players so games that they play don't count as
"genuine."

3. XG2's verdict isn't clearly wrong, because we can't prove that it's
wrong.

---
Tim Chow