How would I do against Mochy in a 13 point match?

[Paul]

Recently I speculated on my winning chances against Mochy in a 13 point match. So I'm going to experiment by playing XG at world-class level. I think this is a reasonably close proxy. Presumably, the bot is stronger than Mochy. However, on the other hand, Mochy would have more psychological shrewdness.

I will play as many matches as I have time for (which may well be very few) and then report the result. Unfortunately, I don't have nearly enough time to get results of any statistical significance (I'll probably only have time for three or so). I hope I win them all.

Hmmm, Tim would say (correctly) that the experiment is meaningless if I don't predetermine how many matches I play. Well, I won't predetermine it so it will just have to remain meaningless.

I hope to win 4-0 which would be (weak) evidence against the hypothesis that my chances would only be 30%.

Paul

[Bradley K. Sherman]

I'll predict a 17% (1 in 6) win rate for Paul against XG in 13 point
matches. Mochy would have the advantage of realizing that Paul is a
patzer and would alter his play appropriately.

--bks

[Paul]

It sounded pessimistic but you might be about right. I've lost every match so far and I'm on about my fifth.

I'm giving up for now.

Paul

[badgolferman]

Bradley K. Sherman wrote:

>I'll predict a 17% (1 in 6) win rate for Paul against XG in 13 point
>matches. Mochy would have the advantage of realizing that Paul is a
>patzer and would alter his play appropriately.

I'm going to try this with gnubg. Six matches 13 points.

[badgolferman]

Is grandmaster the proper level to use?

[Paul]

This seems clearly a matter of personal choice so it's hard to understand your question. If you mean to ask which gnubg level corresponds to XG world-class, the likely answer is that XG world-class beats gnubg at any level, unless you give gnubg an impractically slow setting. However, as usual for technical bot-oriented questions, Tim Chow is an eminent authority on the matter, whereas I am just spouting ignorant guesses.

I always want the bot to play as strongly as possible, but I don't have the patience to wait more than a minute for almost every move. I play XG at the strongest setting which doesn't keep me waiting interminably. This is world-class and it is also the default setting.

Paul

[michae...@gmail.com]

I posted a new topic titled "Calculating your winning chances with amazing precision" for future reference. You guys might want to have a look.
If you have the patience to play many matches Vs a bot, then always have in mind that the bot never errs on cube.
A practical way to beat it is actually using the cube in a tricky way
a)Never take any cube when you don't have a sure take.
b)Never double unless you are almost certain it's a drop.

On the other hand the bots do take very risky cubes, and they usually lose bad. As soon as you establish a lead in a match then play cautiously. The more the bot loses the more it will play riskier and riskier to make up the difference. Be patient it it will eventually give up. ;-)

[Tim Chow]

On Thursday, December 25, 2014 4:19:58 PM UTC-5, badgolferman wrote:
> Is grandmaster the proper level to use?

Either world-class or grandmaster would be fine. The difference should not be detectable over the course of only six matches. Similarly, XG has various levels but for the purposes of this experiment, anything stronger than its default setting will be basically indistinguishable from its default setting (3-ply).

I have not played any 13-point matches against XG or GNU. I have played a large number of 7-point matches. The computer is about a 5-to-3 favorite against me (slightly more than that when I'm tired/steamed/playing fast and slightly less than that when I'm rested/calm/playing slowly). I don't really know how well I would fare in 13-point matches---worse, obviously, but I don't trust the formulas that are commonly used.

I suspect that Mochy's record against Paul would be slightly worse than XG's, provided enough matches were played for the luck to average out. Mochy does not do a lot of adjusting for the opponent. It's possible that if Mochy and Paul played enough, then Mochy would start learning specifically what Paul's weaknesses are, and would be able to take advantage of that.

If Paul does worse against Mochy than against XG, then I suspect the differences will have more to do with the differences in the way Paul plays than in the ability of Mochy to adjust against a weak opponent. For example, Paul might play over-confidently against Mochy, or maybe the matches might take place in a setting that is unfamiliar and distracting for Paul.

---
Tim Chow

[badgolferman]

gnubg grandmaster six 13-point matches

Match 1 = 12-14 (loss)
Match 2 = 3-13 (loss)
Match 3 = 10-14 (loss)
Match 4 = 2-15 (loss)
Match 5 = 15-12 (win)
Match 6 = 4-16 (loss)

The blowouts were largely a result of large initial leads. Two of them
started out 4-0, one started out 6-0 and the other 8-0. The ones that
were close were neck and neck or I won the first few games.

I guess I need to learn how to play from far behind better. I think I
take too many chances or accept bad cubes.

[michae...@gmail.com]

On Saturday, December 27, 2014 2:48:31 AM UTC+2, Tim Chow wrote:
> I have not played any 13-point matches against XG or GNU. I have played a large number of 7-point matches. The computer is about a 5-to-3 favorite against me (slightly more than that when I'm tired/steamed/playing fast and slightly less than that when I'm rested/calm/playing slowly). I don't really know how well I would fare in 13-point matches---worse, obviously, but I don't trust the formulas that are commonly used.

That's a very good opportunity to check the formulae. If you have a profile on XG then
goto Players/See profile results/Results
Write down the values of your checker Play and your cube Play, then multiply by 1000
and do the calculations mentioned in the other topic. Or just give me the values and i will just input them on my program and give you the results.
Your checker play should be around -0.010 I think.

I don't have any reliable records Vs bots so I just ordered GNU to analyze my last 30 7 pointers (in which i was doing a lot of experimentation anyway) it got me to -13.8 on checker play and -39.39 on cube, then I used the formulae, they give me 35.3% Vs a bot on 7 pointers, and 32% on 13 pointers...

[michae...@gmail.com]

I should point out that the formulae assume equal luck.
I will update the other topic to account for unequal luck.

[Bradley K. Sherman]

badgolferman <REMOVETHISb...@gmail.com> wrote:
> ...

>I guess I need to learn how to play from far behind better. I think I
>take too many chances or accept bad cubes.

When I play casually against the bot I definitely take too
many cubes. Why not? That's why money is such an important
part of backgammon, it keeps one honest. We had an office
game where a rotating crew of about ten different people
played for a dime (i.e. ten cents) a point. Those games
were taken deadly seriously. If there had been no money
involved, the game would have died in a fortnight. As it
was it lasted for several years.

--bks

[Tim Chow]

On Saturday, December 27, 2014 8:29:15 AM UTC-5, michae...@gmail.com wrote:
> That's a very good opportunity to check the formulae.

Yes, you're right, but for the luck to even out, I feel that I'd have to play at least a hundred 13-point matches, and I don't feel like doing that any time soon.

---
Tim Chow

[Tim Chow]

On Saturday, December 27, 2014 10:06:22 AM UTC-5, Bradley K. Sherman wrote:
> When I play casually against the bot I definitely take too
> many cubes. Why not? That's why money is such an important
> part of backgammon, it keeps one honest. We had an office
> game where a rotating crew of about ten different people
> played for a dime (i.e. ten cents) a point. Those games
> were taken deadly seriously. If there had been no money
> involved, the game would have died in a fortnight. As it
> was it lasted for several years.

What you say is true of most people. It happens not to be true of me. I play as accurately as I can when I play the bot.

I do also play in a chouette with colleagues. Most of my colleagues require money to get them to take the game seriously. We play for high stakes by your standards: 50 cents a point. This is not high enough to make any material difference to any of us---people occasionally lose upwards of $50 in a single night without batting an eyelid, because we don't play that often---but it still gets my colleagues to try their best. As for me, I don't particularly like playing for money, but I tolerate it so that I can get a good game. Losing sums of money of this order of magnitude has no emotional impact on me. In fact, the times I get most upset in backgammon are when I'm at home playing against the computer, and I see the analysis and see how I've made numerous bone-headed mistakes. This can sometimes cause me to yell at myself, pull at my hair, pound my own head, etc. I definitely get far more upset about this than I do about losing $50 in a chouette.

---
Tim Chow

[Paul]

On Thursday, December 25, 2014 4:54:25 PM UTC, Bradley K. Sherman wrote:

Bradley,

This is a massive underestimate. Today, for example, I've won the vast majority of matches -- something like 6-3 to me, today.
Unfortunately, I got crushed 5-0 at the beginning, and I thought this was typical, and didn't trust myself enough, so I haven't been keeping score. I think I'm around 40% but it's hard to be an unbiased judge. Certainly 17% is ridiculous.

Paul

[Bradley K. Sherman]

Paul <peps...@gmail.com> wrote:
>On Thursday, December 25, 2014 4:54:25 PM UTC, Bradley K. Sherman wrote:

>> ...

>> I'll predict a 17% (1 in 6) win rate for Paul against XG in 13 point
>> matches. Mochy would have the advantage of realizing that Paul is a
>> patzer and would alter his play appropriately.

> ...

>This is a massive underestimate. Today, for example, I've won the vast
>majority of matches -- something like 6-3 to me, today.
>Unfortunately, I got crushed 5-0 at the beginning, and I thought this
>was typical, and didn't trust myself enough, so I haven't been keeping
>score. I think I'm around 40% but it's hard to be an unbiased judge.
>Certainly 17% is ridiculous.

> ...

3 out of 11 so far? I don't think we're at "massive" yet. I will
be very surprised if you can remain above 30% in the long run.

--bks

[Tim Chow]

On Monday, December 29, 2014 7:24:28 PM UTC-5, Bradley K. Sherman wrote:
> Paul

What? Ignoring the intermediate games that Paul failed to keep track of, that's 6 out of 11, not 3 out of 11.

---
Tim Chow

[Bradley K. Sherman]

In article <af85f27d-9b8c-4a0b...@googlegroups.com>,

Oops! Too much champagne. However I stand behind the ultimate sentence.
66%? Preposterous over the long run.

--bks

[Tim Chow]

On Monday, December 29, 2014 7:05:30 PM UTC-5, Paul wrote:
> I think I'm around 40% but it's hard to be an unbiased judge.

I suspect that the top players in the world are probably around 40% against the computer in a 13-point match. Maybe a little bit more than that but not much.

---
Tim Chow

[Bradley K. Sherman]

By the way, my calculation was based on my own score against
gnubg in five-pt matches. I assumed that Paul would win 40%
of 5-pt matches and figured that a 13-point match was something
like winning two 5-point matches in a row (.4 * .4 = .16).

--bks

[peps...@gmail.com]

0:5 + 6:3 = 6 out of 14 not 6 out of 11.

Although this is still a very small sample, I did believe that I could maintain something like this in the long run. However, if it's equivalent to the best players in the world, then that means that I would not be able to maintain it.

I'm not an expert.

Paul

[Paul]

Where do you get 66% from? I don't think I'll ever play enough games to get a statistically large enough sample. I really think that a 17% win rate is implausibly low, based on my limited experience. However, it's also implausible that my win rate would match the best players in the world. 30% certainly sounds plausible. I would hope it's higher than that, but that might be just vanity.

Paul

[Paul]

Based on contributions to the newsgroup, your understanding of the positions and ability to understand the quiz positions is stronger than mine. I would therefore seriously doubt that I'm better than you at backgammon.

However, you did say that you take too many cubes etc. against bots, so it seems that you're not really trying your best. So maybe your own 40% doesn't reflect your true strength. If you can play at your own full strength against the bot, and report your representative scores, I would think that I wouldn't better those in the long run.

Paul

[Paul]

This is very surprising to me, indeed. I would have guessed something like 48%.

Please could you provide some evidence?

Thanks,

Paul

[Paul]

Winning a 13 point match is not like winning two 5 point matches in a row. With that logic, XG would only beat itself 25% of the time in a 13 point match, instead of 50%. Probably, this is the (faulty) reasoning that is leading you to an underestimate.

Paul

[michae...@gmail.com]

The guy who owns greedygammon had Othello (No 34? Giant) playing the bots there perhaps he could provide some evidence.

[mu...@compuplus.net]

On Friday, December 26, 2014 5:48:31 PM UTC-7, Tim Chow wrote:

> Either world-class or grandmaster would be fine. The difference
> should not be detectable over the course of only six matches.
> Similarly, XG has various levels but for the purposes of this
> experiment, anything stronger than its default setting will be
> basically indistinguishable from its default setting (3-ply).

I keep hearing this and it's very interesting to me because I
seem to do better against gnu 4ply than against grandmaster and
supremo or even world class. I hope I'll get to the bottom of
this someday...

In the meantime, since I first read this thread, I started an
experiment of 10 matches of 13-points against gnu 4ply, using
the date (Dec 25th) as part of the seeds, (i.e. 122501, 122502,
etc.) while I was playing the 10th game my PC froze, so I decided
to reply to some articles sooner than waiting for the end result
of my experiment. In 9 matches I am 6-3 ahead, so the worst that
can happen is 6-4... (sigh :)

When I'm done, I will post the matches and gnu's analysis so that
you can spew your expert bs on all of that... ;)

MK

[Bradley K. Sherman]

Paul <peps...@gmail.com> wrote:
>On Tuesday, December 30, 2014 1:43:19 AM UTC, Bradley K. Sherman wrote:

>> ...

>> By the way, my calculation was based on my own score against
>> gnubg in five-pt matches. I assumed that Paul would win 40%
>> of 5-pt matches and figured that a 13-point match was something
>> like winning two 5-point matches in a row (.4 * .4 = .16).
>
>Winning a 13 point match is not like winning two 5 point matches in a
>row. With that logic, XG would only beat itself 25% of the time in a
>13 point match, instead of 50%. Probably, this is the (faulty)
>reasoning that is leading you to an underestimate.

That's not exactly what I said. I was just saying that for
a much weaker player to win a 13-point match against a
stronger player was "something like" winning two 5-point
matches in a row. What is the appropriate adjustment for
the longer match given that we know the frequency of
wins for the smaller?

--bks

[Paul]

On Tuesday, December 30, 2014 2:31:54 PM UTC, Bradley K. Sherman wrote:
... What is the appropriate adjustment for

> the longer match given that we know the frequency of
> wins for the smaller?
>

Michael seems to have studied the question in detail, and his statistics seem in line with my experience.

I look forward to his answer.

Paul

[mu...@compuplus.net]

On Tuesday, December 30, 2014 5:20:51 AM UTC-7, mu...@compuplus.net wrote:

> In 9 matches I am 6-3 ahead, so the worst that
> can happen is 6-4... (sigh :)

It's 7-3. At 4 gnu 5 me score, it doubled to 8 and
ended up resigning a gammon. Stupid bot... :)

Batch analysis in progress. Results coming up soon.

MK

[Bradley K. Sherman]

FIBS ratings go up with the square root of the length of
the match.

--bks

[Paul]

Ok. Let's use your own performance as a benchmark and let's use your figure of 40% wins against gnu in 5 point matches. Using that, what is the win rate in 13 point matches? I think that 0.4 * 0.4 = 0.16 is much much too low. How does the 40% translate to 13 point matches?

Paul

[Bradley K. Sherman]

Paul <peps...@gmail.com> wrote:
>On Tuesday, December 30, 2014 5:36:26 PM UTC, Bradley K. Sherman wrote:

>> ...

>> FIBS ratings go up with the square root of the length of
>> the match.
>
>Ok. Let's use your own performance as a benchmark and let's use your
>figure of 40% wins against gnu in 5 point matches. Using that, what is
>the win rate in 13 point matches? I think that 0.4 * 0.4 = 0.16 is
>much much too low. How does the 40% translate to 13 point matches?

I think you will win less than 30% of your 13-point matches against XG
at its highest rating. That would be my money proposition. I think
the true record will be less than 25% (over the long run, of course).

This is not based on some sort of rigorous mathematical argument,
just intuition based on my own results against gnubg with the
assumption that gnubg < XG AND bks > Paul.

--bks

[Paul]

Ok, we can do a prop based on that (but not a money prop).
If I achieve more than 30%, you agree to read Infinite Jest by David Foster Wallace. If I achieve less than 30%, I will give you my opinion as to whether Infinite Jest is worth reading.

Deal?

Paul

[Bradley K. Sherman]

Paul <peps...@gmail.com> wrote:
>On Tuesday, December 30, 2014 5:49:51 PM UTC, Bradley K. Sherman wrote:

>> ...

>> I think you will win less than 30% of your 13-point matches against XG
>> at its highest rating. That would be my money proposition. I think
>> the true record will be less than 25% (over the long run, of course).
>>
>> This is not based on some sort of rigorous mathematical argument,
>> just intuition based on my own results against gnubg with the
>> assumption that gnubg < XG AND bks > Paul.
>>
>
>Ok, we can do a prop based on that (but not a money prop).
>If I achieve more than 30%, you agree to read Infinite Jest by David
>Foster Wallace. If I achieve less than 30%, I will give you my opinion
>as to whether Infinite Jest is worth reading.
>
>Deal?

I never welch on bets, Paul, but I can no longer read fiction.
As a boy, I could read several novels in a day, but these days
I just cannot get started. Maybe we can find a better proposition.
I'll tell you what: You play a new set of ten 13-pt matches. If
you win four or more I will send you $10 (U.S.). All you have to
do is put up your time and be honest.

--bks

[Paul]

Sorry, no deal. If I have the time, I will play ten matches and report the score. However, I won't owe you anything, regardless of my performance. Therefore, I don't expect anything from you if I get a good result.

Thanks,

Paul

[Paul]

Bradley,

Sorry on rereading it, your prop is that I wouldn't owe you anything even if I lost -- I just "put up my time".

I'll try and play 10 matches. However, if I do well, I would rather _not_ receive money from you.

Paul

[Tim Chow]

I don't have direct evidence of the sort that I would wager money on, but the estimate comes from the following numbers that people seem to bandy about a lot.

1. 3 PR points is about 100 Elo rating points.

2. Top players have a PR of about 3, XG has a PR of about 0.

3. The FIBS rating formula predicts that a 100 Elo rating-point advantage translates into about a 40% win rate for the underdog in a 13-point match.

---
Tim Chow

[Bradley K. Sherman]

Tim Chow <tchow...@yahoo.com> wrote:
>
>3. The FIBS rating formula predicts that a 100 Elo rating-point
>advantage translates into about a 40% win rate for the underdog in a
>13-point match.
>

What would the win rate be for 5-point matches?

--bks

[Tim Chow]

On Tuesday, December 30, 2014 6:57:34 PM UTC-5, Bradley K. Sherman wrote:
> What would the win rate be for 5-point matches?

I'm using the calculator here:

http://www.netadelica.com/bg/fibscalc.html

Its answer is 44%.

Playing around, I find that it thinks that if your win rate is 40% for 5-point matches, then it estimates your rating difference at around 150 Elo points. Then if you ask it what the chances are for a 150-Elo underdog in a 13-point match, it suggests 35%.

As I said, I don't really trust these formulas but it does suggest that your estimate of winning a 13-point match as being about the same as winning two consecutive 5-point matches seems to be off. Roughly speaking, I think the problem is that to win two consecutive 5-point matches, you have to be ahead in both "halves," whereas you can win a 13-point match by being slightly behind in the first half and catching up in the second half, or vice versa, and that counts for a significant percentage of your wins.

---
Tim Chow

[Bradley K. Sherman]

Tim Chow <tchow...@yahoo.com> wrote:
> ...

>Playing around, I find that it thinks that if your win rate is 40% for
>5-point matches, then it estimates your rating difference at around 150
>Elo points. Then if you ask it what the chances are for a 150-Elo
>underdog in a 13-point match, it suggests 35%.
>
>As I said, I don't really trust these formulas but it does suggest that
>your estimate of winning a 13-point match as being about the same as
>winning two consecutive 5-point matches seems to be off. Roughly
>speaking, I think the problem is that to win two consecutive 5-point
>matches, you have to be ahead in both "halves," whereas you can win a
>13-point match by being slightly behind in the first half and catching
>up in the second half, or vice versa, and that counts for a significant
>percentage of your wins.

Yes, the evidence is mounting that I'm an idiot, not that
there was much question about it. I certainly would have
thought that even the difference between 5 and 9 pts would
have been greater than that!

--bks

[Paul]

One consideration is that a world-class player almost certainly knows the entire MET for 5 point matches (and how to use it). However, I don't think most of them would know the MET for 13 point matches, but would be using an approximation.
I don't know how significant this loss of MET is, but I doubt that any formula takes account of this.
If this factor is important, it could be that the discrepancy between 5 point and 13 point matches is higher than what the formula says.

Paul

[Tim Chow]

On Wednesday, December 31, 2014 8:38:18 AM UTC-5, Paul wrote:
> One consideration is that a world-class player almost certainly knows the
> entire MET for 5 point matches (and how to use it). However, I don't think
> most of them would know the MET for 13 point matches, but would be using an
> approximation.
> I don't know how significant this loss of MET is, but I doubt that any
> formula takes account of this.
> If this factor is important, it could be that the discrepancy between
> 5 point and 13 point matches is higher than what the formula says.

What you say is correct, but my personal belief is that knowing the MET exactly has very little to do with it. I would expect that over the course of twenty 13-point matches, at most once would one of the world's top players make an error that could be clearly attributed to not knowing a MET value accurately enough. And when I say "at most once" I think that it's more likely than not that there would be zero errors of this type.

And this is assuming that we're measuring the player's performance by their error rate, rather than their win rate. To maximize their win rate, they should probably be using a fish MET. I suspect that the top players don't know a 5-point fish MET significantly better than a 13-point fish MET.

---
Tim Chow

[Paul]

Since this challenge, I have now played exactly ten games with XG. I won three of these games so I narrowly failed your challenge. This challenge meant something to me even though I would have refused the money anyway.

Thanks for the fun.

Paul

[Paul]

Sorry, by "games", I mean 13-point matches (as you hopefully guessed).

Paul

[Bradley K. Sherman]

Paul <peps...@gmail.com> wrote:
> ...

>Since this challenge, I have now played exactly ten games with XG. I won
>three of these games so I narrowly failed your challenge. This
>challenge meant something to me even though I would have refused the
>money anyway.

> ...

If you and Tim had kept your (virtual) mouths shut you could
have gotten a proposition of 3 or more rather than 4 or more.

The question of the effect of match length is fascinating.
My intuition needs a serious adjustment.

--bks