info
Google Groups không còn hỗ trợ đăng ký sử dụng hoặc đăng nội dung mới trên Usenet. Bạn vẫn có thể xem nội dung cũ.
Dismiss
Supernatural Extreme Extraterrestrial buggy bots useless for experiments.
35 lượt xem
Chuyển tới thư đầu tiên chưa đọc
MK
03:58:31 12 thg 9, 202212/9/22
đến
On August 16, 2022 at 1:06:07 AM UTC-6, Axel Reichert wrote:
> I have done so about two years back in a different context.
> No need to program a single line, just have GNU Backgammon
> play itself, say, "Expert" against "Beginner". And since "Beginner"
> is still far better than "Random", set the "noise" ("Advanced
> Options") to 1, the maximum value. Have fun watching!
> But it seems (also for "random" checker play) that a noise
> value of 1 is not enough and does not result in completely
> random play, be it cube or checkers.
So, with this understanding, I decided to run some quick and
dirty experiments which would be better than nothing and, in
fact, would serve my purpose almost just as well by allowing
me to compare cube error rates with actual win rates.
In CLI mode, Gnubg can run a session without further human
interaction but then you can't stop it reliably without losing an
entire session. If you can though, you can save it in SGF format
and print out statistics, etc.
In GUI mode, it asks you to press "OK" after each game, which
I solved by creating a tiny program to keep sending it "ENTER"
automatically for a set number of times, i.e. 100, 500, 1000, so
that I could walk away from it and later save the sessions in
manageable size files.
I set one player to Grandmaster checker and cube, one player
to Grandmaster checker and cube level to below beginner with
maximum noise. After letting it run for a dozen games, I checked
the stats to see how things were going. What did I see..? This..!
===================================================
gnubg random
Chequer Play Statistics:
Total moves 254 254
Unforced moves 219 221
Unmarked moves 254 254
Moves marked doubtful 0 0
Moves marked bad 0 0
Moves marked very bad 0 0
Error total EMG (Points) -0.000 ( -0.000) -0.000 ( -0.000)
Error rate mEMG (Points) -0.0 ( -0.000) -0.0 ( -0.000)
Chequerplay rating Supernatural Supernatural
Cube Statistics:
Total cube decisions 81 98
Close or actual cube decisions 24 37
Doubles 4 10
Takes 1 7
Passes 3 3
Missed doubles below CP (EMG (Points)) 0 0
Missed doubles above CP (EMG (Points)) 0 0
Wrong doubles below DP (EMG (Points)) 2 (-2.602 ( -6.559)) 7 (-3.566 ( -3.566))
Wrong doubles above TG (EMG (Points)) 2 (-0.567 ( -1.082)) 0
Wrong takes (EMG (Points)) 0 0
Wrong passes (EMG (Points)) 0 1 (-1.663 ( -3.327))
Error total EMG (Points) -3.169 ( -7.641) -5.229 ( -6.893)
Error rate mEMG (Points) -132.0 ( -0.318) -141.3 ( -0.186)
Cube decision rating Awful! Awful!
===================================================
Whaah!? However, if you set both players to preset/default levels
without fiddling with noise, etc. it works as expected, like this:
===================================================
gnubg random
Chequer Play Statistics:
Total moves 180 178
Unforced moves 141 151
Unmarked moves 180 178
Moves marked doubtful 0 0
Moves marked bad 0 0
Moves marked very bad 0 0
Error total EMG (Points) -0.000 ( -0.000) -0.000 ( -0.000)
Error rate mEMG (Points) -0.0 ( -0.000) -0.0 ( -0.000)
Chequerplay rating Supernatural Supernatural
Cube Statistics:
Total cube decisions 48 55
Close or actual cube decisions 28 25
Doubles 7 7
Takes 3 6
Passes 4 1
Missed doubles below CP (EMG (Points)) 0 0
Missed doubles above CP (EMG (Points)) 0 0
Wrong doubles below DP (EMG (Points)) 0 0
Wrong doubles above TG (EMG (Points)) 0 0
Wrong takes (EMG (Points)) 0 0
Wrong passes (EMG (Points)) 0 0
Error total EMG (Points) -0.000 ( -0.000) -0.000 ( -0.000)
Error rate mEMG (Points) -0.0 ( -0.000) -0.0 ( -0.000)
Cube decision rating Supernatural Supernatural
===================================================
Apparently, it applies the noise selection to both players even if
the settings panel shows everything as selected. So, you end up
with a weird combination of Grandmaster cube with maximum
noise which is also "Awful!" but less "Awful!" than Beginner cube
with maximum noise... :)
How can you guys ever trust these pieces of garbage bots with
anything is simply beyond me. You must be some pious, devout
believer sorts of "bot fearing folks"... ;)
And what about XG..? With XG you can't even begin to try running
such experiments because it doesn't allow you to select separate
checker and cube levels for the players.
Why would you want to do such things anyway, right? Just learn
what the good bot teaches you...
I must admit I often envy you folks. No brains, no headaches. :)
Well, okay, I'll compromise. Lesser brains, milder headaches...
MK
> I have done so about two years back in a different context.
> No need to program a single line, just have GNU Backgammon
> play itself, say, "Expert" against "Beginner". And since "Beginner"
> is still far better than "Random", set the "noise" ("Advanced
> Options") to 1, the maximum value. Have fun watching!
> But it seems (also for "random" checker play) that a noise
> value of 1 is not enough and does not result in completely
> random play, be it cube or checkers.
So, with this understanding, I decided to run some quick and
dirty experiments which would be better than nothing and, in
fact, would serve my purpose almost just as well by allowing
me to compare cube error rates with actual win rates.
In CLI mode, Gnubg can run a session without further human
interaction but then you can't stop it reliably without losing an
entire session. If you can though, you can save it in SGF format
and print out statistics, etc.
In GUI mode, it asks you to press "OK" after each game, which
I solved by creating a tiny program to keep sending it "ENTER"
automatically for a set number of times, i.e. 100, 500, 1000, so
that I could walk away from it and later save the sessions in
manageable size files.
I set one player to Grandmaster checker and cube, one player
to Grandmaster checker and cube level to below beginner with
maximum noise. After letting it run for a dozen games, I checked
the stats to see how things were going. What did I see..? This..!
===================================================
gnubg random
Chequer Play Statistics:
Total moves 254 254
Unforced moves 219 221
Unmarked moves 254 254
Moves marked doubtful 0 0
Moves marked bad 0 0
Moves marked very bad 0 0
Error total EMG (Points) -0.000 ( -0.000) -0.000 ( -0.000)
Error rate mEMG (Points) -0.0 ( -0.000) -0.0 ( -0.000)
Chequerplay rating Supernatural Supernatural
Cube Statistics:
Total cube decisions 81 98
Close or actual cube decisions 24 37
Doubles 4 10
Takes 1 7
Passes 3 3
Missed doubles below CP (EMG (Points)) 0 0
Missed doubles above CP (EMG (Points)) 0 0
Wrong doubles below DP (EMG (Points)) 2 (-2.602 ( -6.559)) 7 (-3.566 ( -3.566))
Wrong doubles above TG (EMG (Points)) 2 (-0.567 ( -1.082)) 0
Wrong takes (EMG (Points)) 0 0
Wrong passes (EMG (Points)) 0 1 (-1.663 ( -3.327))
Error total EMG (Points) -3.169 ( -7.641) -5.229 ( -6.893)
Error rate mEMG (Points) -132.0 ( -0.318) -141.3 ( -0.186)
Cube decision rating Awful! Awful!
===================================================
Whaah!? However, if you set both players to preset/default levels
without fiddling with noise, etc. it works as expected, like this:
===================================================
gnubg random
Chequer Play Statistics:
Total moves 180 178
Unforced moves 141 151
Unmarked moves 180 178
Moves marked doubtful 0 0
Moves marked bad 0 0
Moves marked very bad 0 0
Error total EMG (Points) -0.000 ( -0.000) -0.000 ( -0.000)
Error rate mEMG (Points) -0.0 ( -0.000) -0.0 ( -0.000)
Chequerplay rating Supernatural Supernatural
Cube Statistics:
Total cube decisions 48 55
Close or actual cube decisions 28 25
Doubles 7 7
Takes 3 6
Passes 4 1
Missed doubles below CP (EMG (Points)) 0 0
Missed doubles above CP (EMG (Points)) 0 0
Wrong doubles below DP (EMG (Points)) 0 0
Wrong doubles above TG (EMG (Points)) 0 0
Wrong takes (EMG (Points)) 0 0
Wrong passes (EMG (Points)) 0 0
Error total EMG (Points) -0.000 ( -0.000) -0.000 ( -0.000)
Error rate mEMG (Points) -0.0 ( -0.000) -0.0 ( -0.000)
Cube decision rating Supernatural Supernatural
===================================================
Apparently, it applies the noise selection to both players even if
the settings panel shows everything as selected. So, you end up
with a weird combination of Grandmaster cube with maximum
noise which is also "Awful!" but less "Awful!" than Beginner cube
with maximum noise... :)
How can you guys ever trust these pieces of garbage bots with
anything is simply beyond me. You must be some pious, devout
believer sorts of "bot fearing folks"... ;)
And what about XG..? With XG you can't even begin to try running
such experiments because it doesn't allow you to select separate
checker and cube levels for the players.
Why would you want to do such things anyway, right? Just learn
what the good bot teaches you...
I must admit I often envy you folks. No brains, no headaches. :)
Well, okay, I'll compromise. Lesser brains, milder headaches...
MK
MK
16:26:48 13 thg 9, 202213/9/22
đến
On September 12, 2022 at 1:58:31 AM UTC-6, MK wrote:
> In GUI mode, it asks you to press "OK" after each game, which
> I solved by creating a tiny program to keep sending it "ENTER"
> automatically for a set number of times, i.e. 100, 500, 1000, so
> that I could walk away from it and later save the sessions in
> manageable size files.
In case any of you want to do the same, here is a simple sample
> In GUI mode, it asks you to press "OK" after each game, which
> I solved by creating a tiny program to keep sending it "ENTER"
> automatically for a set number of times, i.e. 100, 500, 1000, so
> that I could walk away from it and later save the sessions in
> manageable size files.
code for it: (Just change the number of times to run and compile)
===================================================
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
int seed;
byte keyp;
HWND xgdd;
int main() {
seed = 1;
keyp = 13;
while (seed < 200) {
while ((xgdd = FindWindow (NULL, "GNU Backgammon - Message")) == 0) {
sleep(1);
}
SetForegroundWindow (xgdd);
keybd_event (keyp, 0, 0, 0);
keybd_event (keyp, 0, KEYEVENTF_KEYUP, 0);
printf ("Game: %d \n", seed);
sleep(1);
xgdd = 0;
seed = seed + 1;
}
exit(0);
}
===================================================
And while at it, for the ones who sneered at my simple sample
code using the rand() function as not ramdom enough, here is
the same code using Mersenne Twister:
===================================================
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
#include <time.h>
int N = 624;
int M = 397;
unsigned long mt[624];
int mti = 624+1;
int seed, die1, die2;
byte keyp;
HWND xgdd;
/* generates a random number on [0,0xffffffff]-interval */
unsigned long genrand(void) {
unsigned long y;
unsigned long mag01[2]={0x0UL, 0x9908b0dfUL};
if (mti >= N) {
int kk;
if (mti == N+1) {
/* initializes mt[N] with a seed */
mt[0] = seed & 0xffffffffUL;
for (mti = 1; mti < N; mti++) {
mt[mti] = (1812433253UL * (mt[mti-1] ^ (mt[mti-1] >> 30)) + mti);
mt[mti] &= 0xffffffffUL; }
for (kk = 0; kk < N - M; kk++) {
y = (mt[kk]&0x80000000UL)|(mt[kk+1]&0x7fffffffUL);
mt[kk] = mt[kk+M] ^ (y >> 1) ^ mag01[y & 0x1UL]; }
for (;kk < N-1;kk++) {
y = (mt[kk]&0x80000000UL)|(mt[kk+1]&0x7fffffffUL);
mt[kk] = mt[kk+(M-N)] ^ (y >> 1) ^ mag01[y & 0x1UL]; }
y = (mt[N-1]&0x80000000UL)|(mt[0]&0x7fffffffUL);
mt[N-1] = mt[M-1] ^ (y >> 1) ^ mag01[y & 0x1UL];
mti = 0;
}
}
y = mt[mti++];
y ^= (y >> 11);
y ^= (y << 7) & 0x9d2c5680UL;
y ^= (y << 15) & 0xefc60000UL;
y ^= (y >> 18);
return y;
}
int main() {
seed = (time(NULL)% 65536);
printf ("Seed %d\n", seed);
while (seed > 0) {
while ((xgdd = FindWindow (NULL, "Dice")) == 0) {
sleep (1);
}
die1 = genrand() % 6 + 1;
//printf ("Rolled: %d ", die1);
//sleep (1);
die2 = genrand() % 6 + 1;
printf ("Rolled: %d %d mti: %d\n", die1, die2, mti);
SetForegroundWindow (xgdd);
keyp = die1 + 48;
keybd_event (keyp, 0, 0, 0);
keybd_event (keyp, 0, KEYEVENTF_KEYUP, 0);
keyp = die2 + 48;
keybd_event (keyp, 0, 0, 0);
keybd_event (keyp, 0, KEYEVENTF_KEYUP, 0);
sleep (1);
xgdd = 0;
}
exit (0);
}
===================================================
As always, you're all welcome but don't say I don't contribute...
MK
MK
15:45:16 14 thg 9, 202214/9/22
đến
On September 12, 2022 at 9:58:31 AM UTC+2, MK wrote:
> I set one player to Grandmaster checker and cube, one player
> to Grandmaster checker and cube level to below beginner with
> maximum noise. After letting it run for a dozen games, I checked
> the stats to see how things were going. What did I see..? This..!
> ===================================================
> ....
> I set one player to Grandmaster checker and cube, one player
> to Grandmaster checker and cube level to below beginner with
> maximum noise. After letting it run for a dozen games, I checked
> the stats to see how things were going. What did I see..? This..!
> ===================================================
> Error rate mEMG (Points) -132.0 ( -0.318) -141.3 ( -0.186)
> Cube decision rating Awful! Awful!
> ===================================================
Thinking that this bug may not be in earlier versions, I went
> Cube decision rating Awful! Awful!
> ===================================================
right to the oldest version I have: 0.15 from (15 years ago).
And the bug has been there since then. What a shame... :(
Not for the bug having never been fixed for that long but for
nobody having discovered it since that time! Apparently I'm
the only one to question the things that I question, look for
answers and find bugs instead.
MK
Philippe Michel
15:35:13 18 thg 9, 202218/9/22
đến
On 2022-09-12, MK <mu...@compuplus.net> wrote:
> In CLI mode, Gnubg can run a session without further human
> interaction but then you can't stop it reliably without losing an
> entire session. If you can though, you can save it in SGF format
> and print out statistics, etc.
>
> In GUI mode, it asks you to press "OK" after each game, which
> I solved by creating a tiny program to keep sending it "ENTER"
> automatically for a set number of times, i.e. 100, 500, 1000, so
> that I could walk away from it and later save the sessions in
> manageable size files.
>
> I set one player to Grandmaster checker and cube, one player
> to Grandmaster checker and cube level to below beginner with
> maximum noise. After letting it run for a dozen games, I checked
> the stats to see how things were going. What did I see..? This..!
> Cube Statistics:
> In CLI mode, Gnubg can run a session without further human
> interaction but then you can't stop it reliably without losing an
> entire session. If you can though, you can save it in SGF format
> and print out statistics, etc.
>
> In GUI mode, it asks you to press "OK" after each game, which
> I solved by creating a tiny program to keep sending it "ENTER"
> automatically for a set number of times, i.e. 100, 500, 1000, so
> that I could walk away from it and later save the sessions in
> manageable size files.
>
> I set one player to Grandmaster checker and cube, one player
> to Grandmaster checker and cube level to below beginner with
> maximum noise. After letting it run for a dozen games, I checked
> the stats to see how things were going. What did I see..? This..!
> Error total EMG (Points) -3.169 ( -7.641) -5.229 ( -6.893)
> Error rate mEMG (Points) -132.0 ( -0.318) -141.3 ( -0.186)
> Cube decision rating Awful! Awful!
> Apparently, it applies the noise selection to both players even if
> the settings panel shows everything as selected. So, you end up
> with a weird combination of Grandmaster cube with maximum
> noise which is also "Awful!" but less "Awful!" than Beginner cube
> with maximum noise... :)
I don't know how you set up the play levels, but I cannot reproduce this
> the settings panel shows everything as selected. So, you end up
> with a weird combination of Grandmaster cube with maximum
> noise which is also "Awful!" but less "Awful!" than Beginner cube
> with maximum noise... :)
kind of result. I get the expected result, with the player using 3 ply
for cube decisions playing "perfectly":
Cube Statistics:
...
Wrong takes (EMG (Points)) 1 (-0.014 ( -0.112)) 0
Wrong passes (EMG (Points)) 0 0
Error total EMG (Points) -0.014 ( -0.112) -0.618 ( -0.618)
Error rate mEMG (Points) -2.0 ( -0.016) -15.8 ( -0.016)
Cube decision rating Supernatural Intermediate
The -0.014 is apparently due to the "error" being a take in a double /
beaver / raccoon / beaver / take sequence instead of beavering
forever. If I play a match instead of a money session the cube error
is exactly 0.
I started gnubg on a linux machine from a terminal, so I get the
sequence of commands corresponding to the GUI choices:
(No game) set player 0 gnubg
(No game) set player 0 chequer evaluation plies 3
(No game) set player 0 movefilter 1 0 0 16 0.320
(No game) set player 0 movefilter 2 0 0 16 0.320
(No game) set player 0 movefilter 3 0 0 16 0.320
(No game) set player 0 movefilter 3 2 0 4 0.080
(No game) set player 0 movefilter 4 0 0 16 0.320
(No game) set player 0 movefilter 4 2 0 4 0.080
(No game) set player 0 cube evaluation plies 3
(No game) set player 1 gnubg
(No game) set player 1 chequer evaluation plies 3
(No game) set player 1 movefilter 1 0 0 16 0.320
(No game) set player 1 movefilter 2 0 0 16 0.320
(No game) set player 1 movefilter 3 0 0 16 0.320
(No game) set player 1 movefilter 3 2 0 4 0.080
(No game) set player 1 movefilter 4 0 0 16 0.320
(No game) set player 1 movefilter 4 2 0 4 0.080
(No game) set player 1 cube evaluation plies 0
(No game) set player 1 cube evaluation prune off
(No game) set player 1 cube evaluation noise 1.000
As far as I can tell the GUI set the parameters correctly and the
evaluation function uses them as expected.
MK
16:55:58 18 thg 9, 202218/9/22
đến
On September 18, 2022 at 1:35:13 PM UTC-6, Philippe Michel wrote:
> On 2022-09-12, MK <mu...@compuplus.net> wrote:
>> I set one player to Grandmaster checker and cube, one
>> player to Grandmaster checker and cube level to below
>> beginner with maximum noise. After letting it run for a
>> dozen games, I checked the stats to see how things were
>> going. What did I see..? This..!
>> Cube Statistics:
>> Error total EMG (Points) -3.169 ( -7.641) -5.229 ( -6.893)
>> Error rate mEMG (Points) -132.0 ( -0.318) -141.3 ( -0.186)
>> Cube decision rating Awful! Awful!
> On 2022-09-12, MK <mu...@compuplus.net> wrote:
>> I set one player to Grandmaster checker and cube, one
>> player to Grandmaster checker and cube level to below
>> beginner with maximum noise. After letting it run for a
>> dozen games, I checked the stats to see how things were
>> going. What did I see..? This..!
>> Cube Statistics:
>> Error total EMG (Points) -3.169 ( -7.641) -5.229 ( -6.893)
>> Error rate mEMG (Points) -132.0 ( -0.318) -141.3 ( -0.186)
>> Cube decision rating Awful! Awful!
> I don't know how you set up the play levels, but I cannot
> reproduce this kind of result. I get the expected result, with
> the player using 3 ply for cube decisions playing "perfectly":
> ....
> reproduce this kind of result. I get the expected result, with
> the player using 3 ply for cube decisions playing "perfectly":
> As far as I can tell the GUI set the parameters correctly and
> the evaluation function uses them as expected.
I tested again using both GUI and CLI. I got similar results.
> the evaluation function uses them as expected.
After I replied to you in bug-gnubg, I ran a 17-point match
using the GUI with the above settings. Below is the result.
What could be wrong? (I also just noticed all luck numbers
are zeros.)
MK
gnubg random
Chequer Play Statistics:
Unforced moves 140 153
Unmarked moves 177 178
Moves marked doubtful 0 0
Moves marked bad 0 0
Moves marked very bad 0 0
Moves marked bad 0 0
Error total EMG (MWC) -0.000 ( -0.000%) -0.000 ( -0.000%)
Error rate mEMG (MWC) -0.0 ( -0.000%) -0.0 ( -0.000%)
Chequerplay rating Supernatural Supernatural
Luck Statistics:
Rolls marked very lucky 0 0
Rolls marked lucky 0 0
Rolls unmarked 0 0
Rolls marked unlucky 0 0
Rolls marked very unlucky 0 0
Luck total EMG (MWC) +0.000 ( +0.000%) +0.000 ( +0.000%)
Luck rate mEMG (MWC) +0.0 ( +0.000%) +0.0 ( +0.000%)
Luck rating None None
Cube Statistics:
Total cube decisions 60 50
Close or actual cube decisions 30 20
Doubles 7 9
Takes 7 2
Passes 2 5
Missed doubles below CP (EMG (MWC)) 0 0
Missed doubles above CP (EMG (MWC)) 0 0
Wrong doubles below DP (EMG (MWC)) 2 (-2.495 (-16.360%)) 7 (-3.492 (-23.949%))
Wrong doubles above TG (EMG (MWC)) 1 (-0.325 ( -3.254%)) 1 (-0.213 ( -0.912%))
Wrong takes (EMG (MWC)) 0 0
Wrong passes (EMG (MWC)) 0 0
Error total EMG (MWC) -2.820 (-19.614%) -3.704 (-24.861%)
Error rate mEMG (MWC) -94.0 ( -0.654%) -185.2 ( -1.243%)
Cube decision rating Awful! Awful!
Overall Statistics:
Error total EMG (MWC) -2.820 (-19.614%) -3.704 (-24.861%)
Error rate mEMG (MWC) -16.6 ( -0.115%) -21.4 ( -0.144%)
Snowie error rate -7.9 ( +0.000%) -10.4 ( +0.000%)
Overall rating Intermediate Casual player
Actual result +50.00% -50.00%
Luck adjusted result +50.00% -50.00%
Luck based FIBS rating diff. n/a
Error based abs. FIBS rating 1971.8 1895.8
Chequerplay errors rating loss 0.0 0.0
Cube errors rating loss 78.2 154.2
MK
17:16:12 18 thg 9, 202218/9/22
đến
On September 18, 2022 at 1:35:13 PM UTC-6, Philippe Michel wrote:
> (No game) set player 1 cube evaluation noise 1.000
Could the noise being deterministic or not make a
difference? In mine that box is unchecked.
MK
0 tin nhắn mới