Building my dream chess computer Part III: Enhanced-UCI-Engines for V3
Dirk
Enhanced-UCI-Engines for V3
Until now I have introduced my „dream chess computer project“ consisting of a PI4 with loudspeaker connected to an (used;-) Revelation 2 and my mobile DGTPI with a PI3b+ inside connected to a DGT bluetooth eBoard and a powerbank as well as my core software development: Picochess V3.
You can read more about the hardware in this thread https://groups.google.com/g/picochess/c/jC-EEwEd15M and more about my PicoChess V3 software in this thread https://groups.google.com/g/picochess/c/HM2Dtzt6gic .
In this new thread I will describe a concept I’ve implemented for my V3 which would allow a more or less easy integration of otherwise very complex new functionality which would normally need a lot of changes in picochess especially for the menu (new options) and in the event control mechanism in order to process the new functionality.
Although in the meantime I now know how to add new menu items/options and how to change/add new events in Picochess, there are some use cases where this really intensiv UI and event changing work even would not be sufficient because the restricted input possibilities of the clock and a really big change in the code design of picochess.
Overall Picochess was and is in the first designed in order to just choose an engine and play against it and this is done in a very -from my point of view- nice and easy way making the playing experience really enjoyable for me.
Thats why it is not so easy to add new more complex functions which do not behave like a normal uci engine: I am thinking for example of the possibility of replaying a game for- and backward, eventually getting a simple analysis for each move or a „guessing the next move“ option for famous games like DJ suggested in this „famous mode“ thread https://groups.google.com/g/picochess/c/TGNBnSIBKF4/m/q6fgIwOhAAAJ or the integration of an online server for playing online games (regardless if all these functions really make so much sense within picochess or should be better done on the PC).
Because of these current enhancement „restrictions“ and my restricted python programming abilities and I found a way for me to add such new functionalities without too much redesign effort in my V3.
My idea is to add a complete new complex functionality which behaves or could behave like an engine via an „enhanced-uci engine“ consisting of…
a) an uci-wrapper around the „engine“ part of the new functionality -> by this we get all possible settings automatically presented in the engine level menu of the clock!
b) eventually a small text file for exchanging additional communication information between the engine and picochess (only when it is not possible via the standard uci protocol)
An additional advantage of this approach is that in case the user wants to add more personalized own options the user could just edit the uci text file for the enhanced uci engine in the same way like the level settings for the engines in picochess . The new options are immediately displayed in the engines options menu - all for free.
Downside is the performance: communication with an „wrapped“ uci engine and processing an extra „communication“ file is of course much slower than a direct implementation within the picochess event handling.
For a „PGN engine“ (see below) this is not much of an issue but for playing online games where the pico and server time shave to be in sync that is unfortunately a bigger issue.
So this is definitely not a perfect or even good way to add new features to picochess but for me it was at least one possibility which I was able to implement (for an official Picochess development source branch I wouldn’t recommend this approach - another reason for this my personal picochess version ;-).
Dirk
Replay of PGN games (semi automatic) in V3
Although I think that this feature is not that important for picochess (because it could be done in a much better way on the PC) I had particular use case for this.
It all started with a really cool (german) radio play: „Nahrungsmittel während der Zeitnotphase“ by Helmut Krausser broadcasted in the German radio in 2005 playing in real time during the last game of a chess tournament. Every time I was listening to the radio play I thought it would be cool if I could „see“ that move sequence in parallel to the radio play.
I found the corresponding pgn here: Link for pgn file
https://de.chessbase.com/portals/3/files/2005/hoerspiel/nahrungsaufnahme.htm
The radio play itself I recorded at that time myself but you can still find it online somewhere eg.
So I wanted to be able to replay this game on my DGT Board while at the same time the radio play was played and all this done with picochess => That was the birth for my PGN engine.
Only some weeks ago I discovered that there has been already an anlog version of this concept around since the 1980:
SciSys Intelligent Chess
With this computer you could replay a game on your TV screen and at the same time listen to audio comments eg from the German grand master Helmut Pfleger.
There is an interesting interview with Frederic Friedel on chessbase (The Origins of the Database with Frederic Friedel: https://en.chessbase.com/post/the-origins-of-the-database-with-frederic-friedel )and Hein Veldhuis published a short paper about this „database“ machine, see http://www.schaakcomputers.nl/hein_v...prototype).pdf
Frederic Friedel with an intelligent chess prototype (picture taken from the video by chessbase)
Nowadays this concept works also perfectly with free chess tutorials from youtube which don’t rely on too intensive showing and playing variants on then screen. We can easily convert them into the .ogg file format so that picochess can play these files.
For this I had to use a different sound system for picochess because I needed addition audio commands (start/pause/continue/restart etc.).
I am now using pygame.mixer, the audio part of the pygame multimedia programming library/framework:
https://www.pygame.org/ , installation via „sudo apt-get install python3-pygame“.
I have implemented a PGN replay engine using the enhanced-uci engine concept with the following features for my V3:
- Replay of a saved or manually imported game without pressing a button (except from starting the replay and moving the pieces hence semi automatic ;-)
- Going forward and backward by just planing the announced move or by taking back the last move(s) which are now displayed on the clock
- Option to play an audio file in parallel (and being able to pause and continue it)
- Replay (or more or less) famous games with guessing the next move option
- Getting a small analysis by pressing the standard depth, score and hint buttons: you can specify the analysis engine in the uci settings of th pgn engine.
Replay of a saved game and a „guessing the move“ game for famous games are now possible via my PGN Replay engine
Semi automatic replay of saved pgn games with hint move/score evaluation by an analysis engine for a specific thinking period (time settings will be changed according to uci file), access to the last saved game, the predefined game slots 1-3 and the all_games game file containing all games you played with picochess (then picochess will select a game in the order you specify in the uci settings: going up, going down or by pure chance).
After loading the PGN engine with corresponding UCI settings for a specific game the PGN header data is displayed.
To start the replay just press the lever and the first move will be shown. After moving the computer move sides are autmatacially switches and the next pgn move is displayed and so on.
If you want to see a score and hint move - just press corresponding buttons on the clock like you would do in a normal game.
The analysis engine and time settings are defined in the uci file of the pgn engine.
With the standard takeback option you can „scroll“ backward and forward through the whole game (in V3 even the next move to be taken back is displayed via clock displayed and LEDs will flash for the Revelation onwers) again (just press the lever to continue forward playing)
If you want to play against an engine at a specific point during the replay of the game: just choose the engine via the clock menu and continue the game against the choosen engine.
https://www.dropbox.com/s/a8gigaie7z1wrtr/PGN_REPLAY_FOOL.mov?dl=0
Furthermore an audio comment file for the pgn file can be specified and will be automatically played during the pgn game replay and can be manually started and stopped during the match. Play & pause the audio comment by just pressing the middle play/pause button of the clock (how logical is this? ;-)
"Guess that move" game option for famous games (for white or black)
Switch "guessing" sides by pressing the lever.
Make your first guess (you can define how often you would like to guess until the correct move is shown by picochess)
If it is correct the next opponent move is played, if you guessed wrong picochess triggers an automatic takeback mode (like for mame engines), showing the move top be taken back. After you have taken back the last move you can either guess a second time (in case you specified this) or picochess will play and announce the correct move)
Additionally the pgn replay guess mode can be used to train opening books when setting an empty pgn file with name 'Book Test' and choosing a specific book which you want to train and guess corresponding moves (eg. create an opening book based on Magnus Carlson and guess what he would play next)
Here I try to guess the next moves of Alick Glennie playing against Turings TUROCHAMP algorithm from 1952 (calculated and executed manually by Alan Turing himself):
https://www.dropbox.com/s/okglrfsyotchmwp/Guess_move_pgn.mov?dl=0
The algorithm has been ported to python as uci engine by Martin (see https://github.com/mdoege/PyTuroChamp) so you can play with picochess against it!
You can read more about this famous historic chess computer game online eg. https://www.chess.com/blog/Ginger_GM/the-history-of-computer-chess-part-2-turochamp
PGN Folder Structure
PGN engine uci settings
jonthorjohn
Jon
Dirk
Scally
Shivkumar Shivaji
--
You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/picochess/1702da21-ae37-4046-addf-9a51cfdaa2c4n%40googlegroups.com.
Dirk
Struggling with „Online-PicoChess“
I have hesitated to write about this topic for a long time but I know that some of you guessed or even knew that I was working on this topic as well. Although nothing has been really finished I thought I just give a status update of my experimental work as this topic belongs also to my enhanced uci engine concept. But don’t expect too much - more on that struggling at the end.
Up to now we the following „remote features“ in picochess:
a) Remote engines - serving an uci engine from a remote PC. This could be also a WB (winboard) instead of an uci engine(together with a WB2UCI converter for Windows for example). Works in V3 in parallel with the local picochess engines (see my post in Building my dream chess computer Part II: The software "Picochess V3")
b) Remote mode (this mode still confuses some users because they don’t know how this mode really works while some think the mode does not work at all): by setting the mode to remote in the picochess menu you can play against someone who is using the picochess webserver. The server can be reached via your local wlan/lan or with the corresponding router settings (port forwarding) via internet as well. You execute your move on the eBoard and your remote opponent enters his moves via the virtual chess board of the webserver (this mode works again from my V1.0 on).
(This picture has been taken with picochess V2.01 where the useless „remote (room) button“ has not been removed in the webserver page (i get rid off this button in my V3)
What is still missing is …
c) the ability to play online on a chess server via the internet
In the past I have stated multiple times that I don’t think that Picochess is the first choice when it comes to online gaming.
Mainly because of the restricted clock UI, the missing board LEDs, the big effort to integrate such a service within picochess because of the current structure and the slow „move input“ process via the board.
For me playing a Blitz or even Bullet game on my tablet is way more comfortable and enjoyable.
Furthermore PicoChess lives from the fact that it is quiet easy to use in every aspect I think I have tried to follow this general rule in my V3 like the automatic takeback option in mame or the PGN engine when guessing moves or the tutor move hint by lifting up & setting down a piece etc.)
Online server integration means almost for sure that we loose some of these benefits with picochess (when done wrong at least ;-) unless you just rely on an additional device where you do all the settings and create challenges on the servers web site etc. My approach does not need an additional device (although lichess offers a nice way to do this, see the manual seek option I have implemented).
But after experimenting some time now with online chess within picochess I must now revise my opinion - at least in a few aspects:
- LEDs: not so important Although I now have a Revelation II with LEDs I still like to play with my bluetooth eBoard together with my „limited edition“ DGTPI and I don’t miss the LEDs at all with this setup and I can execute the engine’s move using move voice announcements almost as fast as on the revelation 2 with LEDs.
- And of course there is more than just bullet and blitz online gaming - playing a 15 or 30 minutes game with a friend online via the board can be a nice experience as well and makes fun like the blitz games on the tablet.
- Regarding the effort to integrate an online gaming server into picochess: now my „enhanced uci-engine concept“ comes into play similar to my PGN engine (yes, you might have guessed it ;-) which allows an integration of „online engines“ with much less effort while at the same time the online configurations fits nicely to the Picochess UI: easy to use and not too complex (I will talk about the disadvantages in a second)
Keep in mind that all the things you read in this post are not finished by any means - just basic prototypes (no processing of game messages like resigning or draw offers, promotion of other pieces than queen etc., and it is not yet tested at all )…
I started with the integration of the Free Internet Chess Server (FICS) because it was rather straight forward to implement when you know the simple protocol via telnet.
First I built an uci wrapper around the FICS protocol so that it behaves (almost) like a normal uci engine. By this I got the menu entries for this service and possible options automatically presented in the (engines) menu and game session specific settings can easily be defined in the uci config file - great!
Next step was to add a basic „framework“ to picochess that makes it possible to easily integrate online engines meaning an extra logic for the „seeking“ phase for finding an online opponent, showing this information in case an opponent has been found and checking the game (end) status etc.
Support for Online Engines in Picochess
I started with the integration of the Free Internet Chess Server (FICS) because it was rather straight forward to implement when you know the simple protocol via telnet.
First I built an uci wrapper around the FICS protocol so that it behaves (almost) like a normal uci engine. By this I got the menu entries for this service and possible options automatically presented in the (engines) menu and game session specific settings can easily be defined in the uci config file - great!
Next step was to add a basic „framework“ to picochess that makes it possible to easily integrate online engines meaning an extra logic for the „seeking“ phase for finding an online opponent, showing this information in case an opponent has been found and checking the game (end) status etc.
The result you can watch in the video I have already posted above:
The world’s first played Picochess Online Game played with Al...
https://www.dropbox.com/s/cew7v5hqakx4l2j/Worlds_First_online_MAtch_with_Al_via_Picochess.mov?dl=0
After a few more refinements I could play without pressing any button against different opponents - here on my DGTPI which is really fun. Here is one of my very fist picochess online video with excellent ;-) live commentary by Dimitri Kamarov:
https://www.dropbox.com/s/ti3bhhtl8yav8el/Online_DGTPI.mov?dl=0
Dirk
The „framework“ of enhanced online uci engines which I added to picochess:
- Switch to Online Mode if online engine is chosen (engine name starts with Prefix 'Online') and automatic login according to the logon-user settings in the uci file.
- Time control settings are taken from the online server challenge and are automatically applied as current time control settings in picochess (will be automatically reset to default time setting which you can specify in the picochess.ini file when you choose a non online engine)
- clock does not starts immediately after first move and online move is automatically played in case of the online opponent plays white (automatic side switch!)
- Player's time already starts with announcement of best move and no longer when the computer move has been done by the user (other than that no real sync with server time has been implemented)
- new online seek triggered in case of 'start new game' event (just replace the pieces in starting position as usual)
- additional online info messages (login, seeking, opponent name, game result)
Here after seeking for a short time an online opponent is found and the opponents name is displayed before the game starts
- the last user move of a game which leads to an end is pushed to the online engine in order to inform the server (this is different when planning with normal engines)
- online player names in pgn file(s) instead of engine or pico user name only
- Online engines won't be saved in picochess.ini as last engine (like the PGN engine) - makes no sense in opinion
- Of course all pico tutor features like move suggestions or blunder warnings are switched off for online play
Unfortunately because of this framework the picochess code got very complex - that's a disadvantage of my "enhanced uci engine concept" compared to a direct integration of the online functionality.
Dirk
The Prototype Online Engines
- very basic FICS (Free Internet Chess Server) Online engine (for this TELNET must have been installed, don’t know anymore if it is already done by default or if I had to do it via „sudo apt-get install telnet“?): faster integration than lichess, lots of seeking options, „listenening“ option (waiting until your opponent challenges you)
Here you can see the „listening“ option so that picochess (logged in twit user ‚TheMeaningOfLife‘ waits until someone (here myself under a different guest account) using best the web client for FICS I know: Mobialia, https://chess.mobialia.com/#run
https://www.dropbox.com/s/r35ekx3a85huqkr/Online_listen.mov?dl=0
- very basic lichess Online engine
Unfortunately it is not so easy for me to write a lichess online engine because of the very restricted API for seeking and matching opponents etc. although that changed with the publication of new BOARD API a little bit (but minimal game duration is 10 minutes here)
As I don’t know that much about web development I started by using a „python wrapper“ library for the lichess Board API (berserk) and applied an uci wrapper around it for my enhanced uci((online) engine framework - so we have a wrapper within a wrapper - no wonder that this approach is kind of slow :-(
We can define basic seek challenges like for the FICS engine completely without any extra PC/tablet or whatever.
But there is a cool possibility which is not possible with the FICS engine: the "manual seek" mode I have implemented for the lichess engine:
Manual seek:
- Start seek from picochess
- Open lichess.org, login with same board api user like Picochess on a PC/tablet/…
- Choose one of the following options in the web client:
a) lichess stockfish engine
Lichess stockfish engine is shown as A.I. Level XXX in picochess:
https://www.dropbox.com/s/hmw0x4v51xbmn75/pico_lichess_stf.mov?dl=0
b) one of many online bots (different engines which user put online as a bot)
https://www.dropbox.com/s/6dmaw47mq829p5g/lichess_bot.MOV?dl=0
c) search for an anonymous online opponent
d) challenge or accept challenge of a friend
Although these prototypes seem quite interesting there is a the kind of showstopper why I paused the current prototype development.
A really big problem is the synchronization of the server and Picochess clock times caused by my enhanced UCI framework solution (which I use for the prototypes) because it is a asynchronous solution so that the design itself allows no correct synchronization of the times. Picochess just waits in its own event loop for the „best move“ event and processing will be always a little behind the moment the actual online server answered.
Furthermore I tried to fix any sync time issues by modifying the picochess code further: meaning start of the clock times should be not immediately with first move, the users time must already run when the opponent move is announced and I spend a new picochess.ini parameter in order to compensate any time loses during the uci communication.
But all that did help very much, especially not for the lichess engine as this is already the slowest one because of the multiple wrapper layers.
So what is the conclusion of all this for me:
Yes, I like playing online with picochess and my eBoards (and I did not expected that before) but only when it is integrated within picochess in a nice, direct and simple way (like with my enhanced uci-engine concept ;-)
By having an additional device in order to open the server webpage and choose the opponent there is also a nice approach (the manual seek option for lichess is really nice) - but only as an option.
Unfortunately my solution also means that server integration is not so tight and fast and that there are severe time synchronization issues that can’t be fully fixed.
That means that...
- my online engines are not recommended for speed gaming like 5 or even less minutes per game (the lichess Board API does not even allow faster games than 10 minutes)
- I would never play with my real user I use for „normal online“ play when using the board with picochess (hence I have created a new user eBoardPlayer user for this case)
- You can’t rely on the times Picochess qre showing cause they will differ from the server times (gap is getting bigger the longer you play (and can be currently more than 1 minute in the end!)
So, will I ever finish the prototypes so that one can play with these time sync issues and delays nevertheless? Hmmm - actually I don’t know at the moment.
But unless someone takes the plunge and implements a more direct integration with less delay and sync issues I will leave my already existing „online framework“ for these online engines in my V3 at least, maybe I will find a good compromise solution and finish the prototypes someday…
Dirk
Dirk
This is the thread for the V3 images: https://groups.google.com/g/picochess/c/SpNFpp2Scw4?pli=1