rollouts, time it takes vs. accuracy of results
brad
blowfish, where the default number of games for rollouts is 1296) for
rollouts for just a casual analysis for a beginning player?
(like for blowfish i set it to 36 and it finishes in like 5 minutes)
am i making a mistake setting it so low? should the floor be 108?
any input is appreciated.
brad
Douglas Zare
> whats the minimum level (number of games i think; i just have
> blowfish, where the default number of games for rollouts is 1296) for
> rollouts for just a casual analysis for a beginning player?
For casual analysis, accept 3-ply evaluations as gospel (I mean what gnu
calls 2-ply). They are already truncated rollouts using the static
evaluations.
When I started, perhaps 90% of the time bot evaluations disagreed with my
play they were right. A short rollout has more of a chance of falsely
favoring the wrong play than it does of finding that a play in which a
weak player is right.
Now that I'm much stronger, of course I use rollouts, since the bots are
only right 87-88% of the time. :) More seriously, it is a very
complicated issue to figure out which rollouts are acceptable. You need
to look at the statistical errors and the systematic errors, and
understanding the systematic errors requires familiarity with the bot and
backgammon in general. In some positions, a wide variety of rollouts will
be accurate, but in some positions no rollouts will give the right
answer.
Douglas Zare
kiwikiwi
Douglas Zare <za...@math.columbia.edu> wrote in message news:<3DA8932B...@math.columbia.edu>...
Douglas Zare
kiwikiwi wrote:
> What kind of positions are not suited for rollout?
That depends on the bot. You don't want the bot to hit a lot of positions it does not understand, hence
will play badly. If you have a cubeless rollout, you also don't want the assumptions of the adjustment
to a cubeful equity to be invalid.
With Snowie, for example, many rollouts of bearoff positions will be inaccurate because Snowie does not
understand the very low cube efficiencies of the late bearoff. Snowie thinks that a 3-roll versus 3-roll
position is almost a take, when it is a big pass (with no effective recubes). I know of no settings of
Snowie rollouts such that 2^5 1^3 versus 1^8 is correctly identified as a money redouble.
Deciding whether to double or not, particularly when you might be too good to double, is very hard to
extract from rollouts of any current bot. They are not good at evaluating the equity of positions that
are too good to double, hence may double too soon rather than play on in a live cube rollout.
In Lamford's book, he claims the last problem is another example. That position had 15 checkers on the
1-3-5 anchors, and 15 checkers on the 6-4-2 points bearing off. There is no reason for bots to
understand such a strange position, and they play it much worse than a human being might.
Douglas Zare
Brad Davis
> Now that I'm much stronger, of course I use rollouts, since the bots are
> only right 87-88% of the time. :)
Which bots and at what settings are you referring to?
Do you have evidence to back this up?
What is your definition of an error?
Thanks
Douglas Zare
Brad Davis wrote:
Did you not get the joke? The ":)" was supposed to make it clear that I was not serious. I am right more
than 13% of the time when I disagree with any bot I have yet encountered.
Douglas Zare
Zorba
> I am right more
> than 13% of the time when I disagree with any bot I have yet encountered.
>
> Douglas Zare
Hm, I thought that was 100%?
--
_
/
_ orba