Retro Engine Position Setup

[RandyR]

Can someone (Dirk? Henri? Dieter?) confirm that position setup for retro engines (that support it) isn't working in v4? I may have forgotten the procedure but I've tried mm2, mm4 and sapphire2. Could be a LUA issue but I don't think so. Thanks. I'll hold off on troubleshooting for now.

Randy

[Dirk]

I am away for the whole week. I can check next week…
Dirk

[Johan Sjöblom]

Randy, no hurry, but just for me to be able to repeat this, are you using the "set position" function that sends the web client position to the picochess backend?

If I know the exact menu function you use, I can probably compare the V3 and V4 codes.

-- Johan

[Randy Reade]

Hi Johan, 

Yes. I'm using the Scan function in the Position menu of the web IU. I'll test a v3 image to make sure I'm doing it correctly. It has been a while since I used the function with retro engines.

Randy

[RandyR]

I can confirm it is working using v3.4 with Mephisto MM V (mm5). I set up a mate-in-3 position (fen: 3b4/8/7p/5K1k/8/6p1/6P1/2R5 w - - 0 1), scanned the board (left other settings the same), and with artwork enabled I was able to confirm the board on the artwork updated correctly. Pressed the 'lever' so MM V would play White and it solved the mate on level 5. I was using a Pi4, my last v3.4 Desktop image (24 Feb 2026), updated Trixie.

Randy

[Johan Sjöblom]

I found the difference in the code. V3 sent the position immediately after scan to the mame engine. I hotfixed V4 to now do the same.
I could not yet determine if I need to also send isready... But at least now its definitely one step closer to the V3 code

Note that V4 also has a web client function "Set Pos" that takes the web client position and sends it to the backend and engine. That one also now uses the same improved mame setposition.

git pull, or update to test at your convenience

-- Johan

[Johan Sjöblom]

and I made one more (2nd) commit eab2859 that also sends isready so now its exactly the same procedure as V3 had

[Johan Sjöblom]

The uci communication behaviour is now the same in V3 and V4:

1. ucinewgame
2. position <scanned/set position>
3. isready
4. wait for readyok

[Johan Sjöblom]

I tested without an eboard:

1. chose Mephisto MM V
2. picked up a game from the game list
3. chose a position from the game for the web client to show by clicking on the  move list window in mid-game somwhere
4. Then I went into the Position menu and took "Set Pos", the position was now sent to the MM V.

After that I could continue the game from that point against MM V.

This was on my Debian development machine, but still, I think it proves that it works...

-- Johan

[Randy Reade]

I was going to ask about that Set Position tile.

I'll double check but the 'isready' isn't needed. If it's sent, the MAME plugins/chessengine/init.lua just sends back a 'readyok' just in case the GUI is waiting for it.

I see from your further messages you've added it anyway. That's probably best. Thanks for the quick fix. I'll test now.

Randy

[RandyR]

The position setup now works but when you press the lever to switch side, the retro engine either does nothing or crashes. Here is the log of the first attempt:

2026-08-17 13:20:03.881 WARNING  picochess - main: invalid parameter given ['--allow-onboard-without-auth=true']
2026-08-17 13:21:10.508 WARNING        web - log_request: 401 POST /channel (10.0.0.239) 0.61ms
2026-08-17 13:21:48.184 WARNING     engine - newgame: mame engine isready ping failed after setup position:
2026-08-17 13:21:48.863   ERROR base_events - default_exception_handler: Exception in callback Protocol.pipe_data_received()
handle: <Handle Protocol.pipe_data_received()>
Traceback (most recent call last):
  File "/usr/lib/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/picochess/venv/lib/python3.13/site-packages/chess/engine.py", line 968, in pipe_data_received
    self._line_received(line)
    ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/opt/picochess/venv/lib/python3.13/site-packages/chess/engine.py", line 981, in _line_received
    self.command._line_received(line)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/opt/picochess/venv/lib/python3.13/site-packages/chess/engine.py", line 1294, in _line_received
    self.line_received(line)
    ~~~~~~~~~~~~~~~~~~^^^^^^
  File "/opt/picochess/venv/lib/python3.13/site-packages/chess/engine.py", line 1445, in line_received
    self.result.set_result(None)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^
asyncio.exceptions.InvalidStateError: invalid state
2026-08-17 13:22:00.191 WARNING picotalker - pico3_sound_player: SoX play timed out after 12.0s for talker/voices/en/christina/newgame.ogg
2026-08-17 13:23:59.649 WARNING     engine - _monitor_force_completion: force move timed out after 2.0s - cancelling hung engine
2026-08-17 13:23:59.650   ERROR  picochess - think: Engine returned Exception when asked to make a move
2026-08-17 13:23:59.650 WARNING     engine - handle_bestmove_0000: engine ping failed with unexpected state: CommandState.NEW
2026-08-17 13:23:59.650 WARNING  picochess - process_main_events: wrong function call [best]! mode: Mode.NORMAL turn: True
2026-08-17 13:24:20.177 WARNING     engine - newgame: mame engine isready ping failed after setup position: CommandState.NEW
2026-08-17 13:24:30.246   ERROR     engine - _engine_task: engine (playing) unexpected error during play_move: CommandState.NEW
Traceback (most recent call last):
  File "/opt/picochess/uci/engine.py", line 872, in _engine_task
    play_response = await self.engine.play(
                    ^^^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/opt/picochess/venv/lib/python3.13/site-packages/chess/engine.py", line 1703, in play
    return await self.communicate(UciPlayCommand)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/picochess/venv/lib/python3.13/site-packages/chess/engine.py", line 997, in communicate
    self.next_command.set_finished()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
AssertionError: CommandState.NEW
2026-08-17 13:24:30.248   ERROR  picochess - think: Engine returned Exception when asked to make a move
2026-08-17 13:24:30.950   ERROR  picochess - process_main_events: engine crashed - game has not ended
2026-08-17 13:24:32.193 WARNING     engine - error_line_received: <UciProtocol (pid=3665)>: stderr >> /opt/picochess/engines/aarch64/mame/mm5_art: line 3:  3666 Killed                  /opt/picochess/engines/mame_emulation/mess $1 -plugin chessengine mm5
2026-08-17 13:24:32.194   ERROR base_events - default_exception_handler: UciPlayCommand failed after returning preliminary result (<Future cancelled>)
protocol: <UciProtocol (pid=3665)>
transport: <_UnixSubprocessTransport pid=3665 returncode=137 stdin=<_UnixWritePipeTransport closed fd=11 closed> stdout=<_UnixReadPipeTransport closed fd=12 closed> stderr=<_UnixReadPipeTransport closed fd=38 closed>>
chess.engine.EngineTerminatedError: engine process died unexpectedly (exit code: 137)

[Johan Sjöblom]

Ah, the isready ping termination caused an error... I hotfixed with one more commit.

Do a git pull and see if it helps

[RandyR]

Not sure if it is related but when setting a position and you select the side to move (side whose turn it is to move), the board gets scanned before pressing the scan tile. This is not the same as v3. And it matters because, when using a DGTPi you make all the selections in order, changing as needed to set the state of the position and the final selection is to scan the board. It also appears to be switching sides when the 'side to move' is set. I thought this was previously fixed.

[RandyR]

Will do...

[Johan Sjöblom]

And since I added the isready ping in the (2) latest commits, if also this version fails, you can try to take the commit fd8f8f2 (2 latest commits missing).

That version only sends position, no isready. If that neither version works (latest master or that one) it means the lever switch-side has a bug.

-- Johan

[RandyR]

Commit ae0eae01 has fixed the issue. Using the lever now correctly switches sides. The 'Side to Move' still does a position scan, though. It should only affect the fen string sent to the engine.

Randy

[RandyR]

Using the DGT clock interface (hamburger menu) still works correctly.

[RandyR]

...meaning, if you switch the 'side to move' no board scan occurs at that time.

[RandyR]

Actually, another scan isn't happening; it's likely due to sending a new fen to the retro engine. Did v3 do this as well? I guess I could check again.

[RandyR]

No. In v3 the position isn't resent to the retro engine when pressing the lever, it just tells the retro engine to move, likely just sending a 'go' command.

[Johan Sjöblom]

OK, yes Randy. This was another thing I had not noticed. The sequence and user flow of "Scan" and "Side to Move" is different in V4. At the moment the set position is sent immediately when you chose "Side to Move".

In V3 you first set "Side to Move" and then scan the board.

I will change the V4 to follow the same sequence and cache/keep the side to move until scan is called for. With one exception: if you are in analysis mode (Mode.PONDER inside the code) and generate a new position on the eboard a new position is detected and set automatically. If that position happens to have the wrong side to move the "Side to Move" needs to send the position. In in this case it should be ok to send it immediately since the board FEN has already been noted. And another thing, in this case the lever button shall also perform the same, but that is already the case. The "Side to Move" sequence will change so that it wont send the position to the engine until scan is called for, EXCEPT if we are in analysis mode, then I will continue to send the position immediately for "Side to Move".

-- Johan

[Randy Reade]

I think the confusion is in the similarity of the terms. In the Position menu the Side to Move determines which colour makes the next move. The lever determines who makes the next move, either the Player or the Engine. And, yes, if you are just analyzing and not playing against an engine then that is a different case.

Randy

--
You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/picochess/28882b73-afdd-45db-9255-36873cd65998n%40googlegroups.com.

[Johan Sjöblom]

Yes, we should probably come up with a better terminology. Lever yes is who, and stated in another way: Which side are you/user playing and which side is the engine playing.

I am thinking that in "Side to Move" we should at least avoid the word "Side"... And in the Game menu "Switch Sides" the Side term is used again...

What about

"Side to Move" --> "Turn" or "Turn to move" or "Move turn" and values to select from are Black or White.

"Switch Sides" (lever) --> "User colour" or "Player colour", or "Switch colour", or...

I guess I also need to look at the lever button in V4, if it sends more to the chess engine than V3, if it should send only "go" uci command for mame engines...

-- Johan

[Johan Sjöblom]

Sequence was a problem only in the web client. The sequence flow of the web client menu is now the same as the clock menu.

[Randy Reade]

I think Side to Move is intuitively correct and likely universally accepted when setting up a position (even the start position). The FEN spec calls it the 'Active Colour" but I don't think that would be recommended in our application.

Switch Sides I think is also best but perhaps the response from PicoChess could be more descriptive. Right now it just announces "Black" or "White" but it does display "Player Black" or "Player White". Perhaps that could be displayed longer on the 'clock' and delay sending the 'go' command to the engine (perhaps 2 or 3 seconds). Not sure if this is hardware dependent - I'm testing on a Pi5. A voice announcement of "Player White (/Black)" to match the text display would also be nice. But I don't see any "Player" .ogg file in English. Or maybe an announcement of "Force engine move" when the lever is pressed.

Or leave things as they are and novice PicoChess users will discover what's happening when the lever is pressed.

Randy

[Randy Reade]

Ah, that's good.

[Johan Sjöblom]

I think I found the V4 problem after having used Scan or Set Pos. The game id, a concept not existing in V3, is incremented in V4 on Scan or Set Pos. The new chess library automatically sends one extra ucinewgame and isready if the game id has changed. So an immediate action after Scan or Set Pos, like pressing the lever (or doing Switch Sides from the menu), will cause the chess library to send those duplicated ucinewgame and isready. The mame engines probably do not like that. I need to figure out a solution to prevent those extra duplications.

[Randy Reade]

Yes. isready isn't an issue with the MAME engines but ucinewgame will reset the retro engine.

--

You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/picochess/3ecc5f45-5743-4735-99bc-9fd4a3634935n%40googlegroups.com.

[Johan Sjöblom]

O dear I say, this was a strange one. I committed a fix for this now, but I understand why it took so long for us to find this. See the very long commit message for technical details.

This is related to the issue we fixed with MAME engines a year ago with Dirk. https://github.com/JohanSjoblom/picochess/issues/72

That issue 72 (last year) showed that MAME engines need ucinewgame immediately when Picochess starts a new game. The new chess library normally delays that command until the first engine move, which is too late for the MAME emulator, so Picochess sends an “eager” ucinewgame itself.

After Set Pos or Scan, however, the chess library did not know that this command had already been sent. When the next action—such as pressing the lever—asked the retro engine to move, the library sends a second ucinewgame and isready. That duplicate sequence will disturb the MAME engine. The latest fix keeps the required eager ucinewgame, but also synchronizes the chess library’s internal state so it knows that the current game has already been announced. The following lever/Switch Sides action should now send only position and go, matching the V3 behavior more closely.

Essentially first summer of 2025 we had to send an extra ucinewgame to get mame engines to work with the new chess library, now a year later we have to try to avoid sending it after a Scan or Set Pos. Both these actions are limited to MAME engines only.

[Johan Sjöblom]

My own test does not work with using lever button immediately after a Set Pos (or Scan) on MM V.

1. Select the historical position (white to move).
2. Use Set Pos.
3. Wait until the position is confirmed.
4. Press Game → Switch Sides.

It was the same test case that I used yesterday, but today I also try the Switch Sides (lever). I get engine error efter it tries to move as a step 5.

[Johan Sjöblom]

The duplicated ucinewgame was not sent, so I seem to have prevented that, but the LUA does not like the same move stack arriving (my guess). I need to add more logging...

[Randy Reade]

But the switch sides should not need to send the position since it is sent after the scan to set up the retro board. It should just send go (and, I guess, movetime) should it not?

[Randy Reade]

I'm only testing with an E-Board at the moment. 

To view this discussion visit https://groups.google.com/d/msgid/picochess/6fad9671-78ed-4b4c-8d94-14d592293eddn%40googlegroups.com.

[Johan Sjöblom]

I was able to prevent the extra ucinewgame, but the LUA does not like the Set Pos duplicated move stack... I assume that Scan will actually work now, but it has to be tested.

I will open an issue for this LUA issue unless I can find a way to prevent the duplicated move stack

[Randy Reade]

I thought it was working correctly if using the clock buttons? I'm not sure why the web UI method can't use the same routine. 🤷🏼‍♂️ But I'm sure you understand the code intricacies better than me. 😊

To view this discussion visit https://groups.google.com/d/msgid/picochess/a4b9a4a9-b232-48b9-9980-ef0ecb9d9a3an%40googlegroups.com.

[Johan Sjöblom]

Yes, while I can prevent a duplicated ucinewgame to be sent (which is what I have done in the master and you are testing at the moment),

... I cannot prevent the duplicated "position startpos moves" sent by the new chess library.

The mame LUA adapter should treat a duplicated position as a no-op:

if received position == lastpos:
    do nothing

The mame LUA adaptor cannot take a duplicated ucinewgame as it resets it, but it should be able to accept an exactly duplicated position.

[Randy Reade]

But there is no position startpos moves when you start with a fen position?? This wouldn't only affect retro engines. 

[Johan Sjöblom]

It started working on my development machine (with an old x86_64 mame setup) when I did this change to LUA:

File:

engines/mame_emulation/plugins/chessengine/init.lua

Diff, changed around line 744

- elseif (last_move == prev_move) then

+ -- Repeating an identical UCI position must be a no-op. This happens when 

+ -- Picochess sets up a PGN position and python-chess sends the same position 

+ -- again immediately before go. 

+ elseif (cmd == lastpos or last_move == prev_move) then

Short explanation:

After Set Pos, Picochess immediately sends the selected PGN position to the MAME engine. When Switch Sides is used before another move is made, python-chess sends the same position again before go.

The MAME Lua plugin did not recognize that the complete move-stack command was identical to the position it had already applied. It therefore replayed the final move from the PGN history, changed its internal side to move, and returned a move for the wrong colour.

The change treats an identical position startpos moves ... command as a no-op. MM V then keeps the correct position and successfully starts playing after Switch Sides.

[Johan Sjöblom]

 Web Position → Set Pos can preserve the selected PGN prefix and its move stack.

[Johan Sjöblom]

So I would recommend that we add the extra "or" with cmd == lastpos to the LUA... Worked nicely for me.

The Scan eboard should work without it, but Set Pos will not.

[Randy Reade]

I think what's confusing (me) is the Set Position name. In v3 the Position menu was so you could set up a position (on the E-Board) and go from there (FEN). 'Set Position' kind of describes that. But, if I understand correctly, the Set Position is used for a point in a complete game (at least up to the point you are interested in) and you want to send the complete move list (position startpos moves...) to the engine. I'm not sure why that's needed. (Maybe for 50-move rules and such). Perhaps the tile/function should be called 'Sync to UI Board'?

If be interested to hear other opinions. I could be out to lunch.

[Randy Reade]

Looking closer I see it does say "Set eboard pos" while Scan says "Get eboard pos". Perhaps that's enough. Especially now that I know what it does.

Randy

[Johan Sjöblom]

Yes, Set Pos might need a better name, if I remember correctly I added it as some kind of reverse of the Scan eboard in commit 919b3c3c just this summer,  instead of reading the eboard it reads the web client move stack and position and sends it to the backend, including to an eboard if one is connected. I thought it would be good to have together with the new explore functionality that allows me to chose any position in a PGN move list.

One possible solution might be that if the engine is a MAME engine it would not send the move stack. But the LUA solution is clean, LUA already checks for getting the same move twice, so it could also check if it gets the same move stack twice. Its a clear no-op situation that cannot harm anything and is according to uci.

Another improved solution could be to split the command into two commands, one that only sends only the web client position to the backend, and another that sends an entire Game up to the chosen point in the game.

And yes, the wording could also be improved...

But at least I got the chess library hack so that the scan eboard works like in V3. And the "Side to Move" flow is now to set side before scanning, but its not sent until scanning is performed, same as in V3.

-- Johan

[Randy Reade]

Hmmm, Position/Set Pos doesn't seem to be working for me. I'm using mm5, set up the previously mentioned Mate-in-3, did a Position/Scan, switched sides with the 'lever' so mm5 would solve the mate. After checkmate, I selected the h8=Q+ move in the PGN list, selected Position/Side to Move to Black, did a Position/Set Posn - the web board reset to that position, but mm5 replaced the pieces to startpos, created a second black Q on h8 and removed the g7 and h7 pawns. It then displayed e2e4 in the MAME chessboard display but didn't make the move. I added your LUA change prior.

I feel this is my mistake. Perhaps I'm using this function incorrectly, once checkmate is made there's no going back or the retro engines don't like what I'm doing.

Randy

[Randy Reade]

A Modern Engine (SF18) seems to work as expected. So it looks like, when using a retro engine, there is a 'position startpos moves ...' being sent rather than a 'position fen'.

One suggestion - when using an eboard, after selecting Position/Set Posn, we should probably get a "Set Pieces" to prompt the player to correct the eboard position and then an "OK" when the eboard matches the web board.

Randy

[Johan Sjöblom]

Yes, it would be convenient to hear an ok message. I will add that. 

Not sure what confused the retro mm engine, maybe the checkmate, or the promotion, or then its sensitive to timing…  the early extra ucinewgame and avoiding the later duplicate ucinewgame (sent by chess library) is what is done differently for the retro engine.

— Johan

[Johan Sjöblom]

Ah... there is some LUA thing that discards a custom startup FEN and replaces it with "startpos" if there is a move stack also sent... Your test started from a custom startup FEN. So for retro engines we do need to separate the "Set Pos" into two commands, one that sends only the FEN, and one that sends the original FEN and the movestack. I dont think we need to fix the LUA but here is what I think happened:

game began from a custom position created by Scan. After several moves, Set Pos preserves both:

• the original custom FEN, and
• the moves leading from that FEN to the selected h8=Q+ node.

Python-chess therefore sends something like:

position fen <custom-mate-position> moves ... h7h8q

But [init.lua (line 593 on my machine)](/opt/picochess/engines/mame_emulation/plugins/chessengine/init.lua:593) currently handles any FEN command containing moves by converting it to:

position startpos moves ... h7h8q

It discards the custom starting FEN. The Lua adapter then tries to replay the mate sequence—including the promotion—from the normal starting position. That explains the bizarre result: pieces returning to start position, pawns disappearing, and an unexpected queen appearing on h8.

At the moment LUA supports the 2 first cases:

1. Custom FEN without history: supported.
2. Standard starting position with history: supported.
3. Custom FEN with history: not correctly supported.

So a retro engine cannot work with case 3 at the moment. As said, not sure we necessarily need to fix this in LUA. Especially if we split the "Set Pos" into a simpler part that only sends the FEN, and another command that sends both the starting FEN and a move stack.

-- Johan

[Johan Sjöblom]

The practical and easy way out would be to:

- For MAME engines Set Pos would only send the FEN position to the chess engine.

... unless we want to update LUA to support case 3 that you tested and gave such a weird result (I had tested 2 actually earlier without knowing it as I had a starting position and full move stack because I chose a game PGN in the middle of a game)

-- Johan

[Dirk]

Yes, with the old chest library in v3 we had to distinguish between the pos fen command with and without the move stack because during a game, the chest library would send from time to time instead of positions startpos and  the pos fen command with the partial move stack. 

Because not all mame engines do support the fen position command (and even then it is way too slow to allow fen position command with move stack during a game) we detect a special set up position scenario for mama engines when sending just the fan position without the move stack.

I don’t know how the new chess library would work in this case if it always sends the startpos command during a game or not. So it would be safe to still support this case.

Am 19.08.2026 um 11:28 schrieb Johan Sjöblom <messi...@gmail.com>:

The practical and easy way out would be to:

--

You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/picochess/72af96c1-548c-4140-8ddd-41d2046d8478n%40googlegroups.com.

[Johan Sjöblom]

Added two more hotfixes.

1. Set Pos only sends FEN if a mame engine is selected. its like scanning the web client position.
   
2. Say OK when eboard becomes correct

— Johan

[Randy Reade]

Great, Johan. I'll try to test today or tomorrow.

Randy

[Randy Reade]

I think we need a "Set Pieces" prompt after selecting Position/Set Pos (if using an e-board) then an OK once the pieces on the board are correct. Currently you only hear it if you move a piece and wait a bit.

Also, I wonder if it is possible, when selecting an earlier position in a previously played out FEN (say, a Mate-in-4), and using the Set Pos (e.g., go back to move 2) that the original FEN is restored along with the (2) moves and this is what is seen in the move list. Currently you can't go back to the original start position (FEN) after you use the Set Pos function.

I think we're almost there.

Randy

[Johan Sjöblom]

Hotfixed master so that it immediately says ”set pieces” when you select Set Pos. When the eboard matches it should say OK.

Not sure if I got your idea correct. You set an original FEN by Set Pos, then you play 4 moves, then choose the position after move nr 2 on web client, and then apply a second Set Pos. Now you no longer have access to the original FEN on a MAME engine.

For a modern engine you would have got the original FEN with a move stack of two moves. On a MAME you now get only the second FEN. If we would fix the LUA custom fen not becoming startpos we could have this feature.

If the use case is to check different mate-in positions maybe we could invent a design where you can store interesting “original FEN” positions, or upload them and then apply Set Pos. Such a feature could also be used to practice “can you find the best move”. We have the games folder, maybe we could have a position folder with stored FENs… If there would be space on the web client we could have a list of FEns to chose from…?

Another type of solution is to have a “position memory” where you could store the original FEN, and then you could ask for it to be returned… 

— Johan

[Randy Reade]

Almost. I could have been clearer. The initial FEN is from the E-Board and scanned in, e.g., mate-in-4. After attempting it, you realize you are not solving it and select an earlier move on the move list, then use Set Pos. Right now you lose the earlier moves in the list and can no longer go back to the initial FEN position.

Yes, those are great enhancements.

Randy

--

You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/picochess/de6e1984-3dc4-4b18-b6e3-4d4c230556f6n%40googlegroups.com.

[Johan Sjöblom]

I checked the save game and load game routine.

I now allow you to save an empty game, and yes, it will save a custom FEN if there is one.

The load game had to be fixed with the same issue as we fixed Scan and Set Pos for a MAME engine. If a MAME engine is selected when you load a game I now send an extra early ucinewgame to the mame engine and avoid the duplicated one.

Therefore you can use the following sequence if you want to be able to go back to the original FEN:

• Scan the original problem position.

  • Original custom FEN becomes the game root.
  • Move stack is empty.

• Play your four moves, or whatever...

  • The backend still knows the original custom FEN.
  • The move stack contains four moves.

• Choose Save Game.

  • The PGN stores the original position in its FEN header.
  • It also stores the four moves.

• Select move two and use Set Pos with MAME.

  • The position after move two becomes a new FEN root.
  • Its move stack is deliberately discarded.
  • The original root is no longer present in the live backend.

• Load the previously saved game.

  • Current V4 loading sees the PGN FEN header.
  • It restores the original custom FEN.
  • It currently ignores the saved moves, leaving an empty move stack.
  • With this latest hotfix, that original FEN is also sent immediately and safely to MAME.

[Randy Reade]

Not my preferred solution. It seems a bit cumbersome having to save and load a position when the base FEN doesn't change. Can we not just store the FEN on initial Scan, save the moves as they are played (move stack) and then just discard the moves in the move stack which happened after you select the move to go back to? i.e.:

Initial Position (FEN)

<play some moves>

FEN + 1. xx yy 2. xx yy 3.xx yy 4. xx ...

<select move 2. .. yy>

<move list shows:>

1. xx yy 2. xx yy

<'Sey Pos' and try again, abandon that line totally so decide to start over at initial FEN. Press 'go to beginning' button below web board. Initial FEN is back on we board, move list is empty, select Set Pos and you are prompted to 'Set Pieces', ...>

Randy

To view this discussion visit https://groups.google.com/d/msgid/picochess/da1637ac-ef2d-4f66-82c3-308b500dd5aan%40googlegroups.com.

[Johan Sjöblom]

Yes, cumbersome, but good to have that possibility also to save positions and actually be able to load them into MAME engines as well... But back to the case. I need to repeat my statement:

--> For a modern engine you would have got the original FEN with a move stack of two moves. On a MAME you now get only the second FEN. If we would fix the LUA custom fen not becoming startpos we could have this feature.

I will try to explain.The reason why we lose the original FEN is because LUA cannot accept a custom FEN and a move list of 2 moves. The LUA error is that it swaps the custom FEN into a starting position. If we fix that, we dont have to throw away the move stack in Set Pos for a mame engine. So its not that we dont save everything, its that the MAME hotfix throws away the 2 moves because the LUA cannot handle them. And because we throw away the 2 moves, the FEN stored and sent to the MAME engine is the FEN position AFTER those 2 moves. If we would fix the LUA we would send the original FEN (its still saved) and 2 moves.

If we dont fix the LUA we have to invent a design where we save the original FEN somewhere else. Thats why I tested the save game path... And had to fix the load FEN + moves for a MAME engine with early ucinewgame also there.

Lets see what we can come up with... Fixing the LUA is probably not so difficult.... If we dont fix it we have to find a store and retrieve design for the original FEN. Thats why I started thinking about a old fashion "position memory” like the calculators had back in the 80s :-)

[Randy Reade]

Ok. Now I see the issue. I had forgotten about the LUA limitation. I suppose an option could be to save the FEN after each move and only send the FEN 'requested' when Set Pos is used, leaving the original move list displayed in the web UI. Or have a routine to generated the FEN from the initial FEN plus moves. Then just send that to the MAME engine.

[Randy Reade]

So, re-reading, the issue is not the FEN being sent, it's sending the initial FEN + move stack. I don't think that is the issue. We want the 'second' FEN sent to the engine. The issue is the display on the web UI. That should remain original FEN + moves. 

To view this discussion visit https://groups.google.com/d/msgid/picochess/a8bc4fd5-ce03-4377-9fac-b58ebb8c3dcdn%40googlegroups.com.

[Johan Sjöblom]

The WEB UI and the picohcess backend and the engine, all three have to have the same position and move list at all times.

As soon as a move is made on the eboard the entire move stack is sent back to the web client and its displayed there, trying to store anything there would therefore be overwritten by the next eboard move.

The issue is therefore that to avoid the LUA bug we do send the second FEN to the backend and the engine, the move stack cannot not sent for a MAME engine, so for a MAME engine we lose the original FEN. A modern engine would receive the original first FEN and a move stack. This is a MAME problem, and I dont see how it can be solved by any storage in the web client or in the backend. There needs to be a user action to both store and retrieve the original FEN for a MAME engine if we dont fix the LUA. And that store and retrieve needs a UX design on how the user would do that. This cannot be solved by simply storing the FEN somewhere if it cannot be retrieved.

I cannot see a solution... yet :-)

-- Johan

[Johan Sjöblom]

Oh, I do see one potential solution. Not sure if its realistic. If I loosen up that first line requirement into:

• Web client and Picochess backend must have the same position and move list.
• Picochess backend and MAME must have the same position.

Then there is a solution that we never send any move stack to a MAME engine. But it would have to be done in all interactions with an engine in the code like Scan and Set Pos, Read Game, Starting analysis, Asking MAME to move, Restarting or updating the engine position, ...

At least one problem is that the MAME engine could never do any move-repetition... and maybe something else. Seems like quite a tough change to make.

[Randy Reade]

I assume the backend knows the starting position and keeps track of the moves. For MAME, since we can't send the initial FEN and subsequent moves, just send the FEN of the chosen position to the engine. That's all you are really interested in at the moment when using Position Setup. Keep the original FEN and moves (up to the new position) in the backend and display it on the web client. Now, everything is in sync and you can carry on with your chess with/against the engine.

If this doesn't make sense due to the way the code works in PicoChess then we should be able to adjust the code to make MAME engines work the same as normal engines. Or, we can adjust the LUA files to accept the initial FEN + move stack, as you mentioned.

If it's not doable, or the end result doesn't justify the effort, by all means this can wait or we can use your save/load workaround. You know the code much better than I.

Randy

[Randy Reade]

Exactly what I was trying to say. 😊

To view this discussion visit https://groups.google.com/d/msgid/picochess/72c099f9-fceb-46e6-9016-ebcdc0dd8bf9n%40googlegroups.com.

[Dirk]

Johan & Randy,

I am somehow lost by your conversation and the problem which occurs…

When we are able to modify only what is sent to the mame engine (only FEN or FEN with movelist) - shouldn’t this solve all problems!? 

When we want to have the mame engine just to se up a specific position we just send the FEN - otherwise (when sending FEN and a move list) the last move is just sent to the engine (assuming that the current game just continues with the next move).  Thats what has been implemembhted in Lua at the moment right!? But we could add additional changes to data which is sent to the mame engine on picochess backend level because the mame engine does not store a move list or does need a move list other than the current move for correct working. 

At least this was the cat ein V3 if I remember correctly.

Should'nt we be able to just save/keep correct  FEN and move list in the webGUI and in picochess  backend and avoid problems in Randys set up by modifying the info (either on ABAP or lua Leven) which is sent to the mame engine!?

When we are sure that during a standard game against a mame engine the new chess lib only sends the position startpos command with move list (of course while no real take back is allowed) we could modify lua script to generally support the position FEN command WITH move list because in this case this command is only sent when a user really sets up a position for a mame engine which should support the position set up feature.

In V3 the chess lib unfortunately also sent the position FEN command during a normal game instead of the startpos commend with complete move list :-(

Dirk

To view this discussion visit https://groups.google.com/d/msgid/picochess/0e345893-2542-411b-bd69-aa573b8b9accn%40googlegroups.com.

[Dirk]

Oh I just saw Johans last comment which goes in the right direction...

[Johan Sjöblom]

The first thing I am going to try is to preserve the move history also for FEN engines. That means fixing the LUA. I have now made one commit that restores the move stack for MAME engines which means that as soon as I can help you repeat my working Lua solution you will have the original FEN and all move list available with a retro engine like for instance mm.

Its the latest commit in master: 0183c7a that restores the move stack for MAME engines. Of course this mainly affects only Set Pos (and loading a custom FEN from a saved game).

In next posts I am going to try to explain my Lua fix.

[Johan Sjöblom]

Here is my working init.lua, and a md file description of what I fixed. I dont know how old my mame emulation folder is... And I have so far only tested on my Debian development machine.
The SHA-256 checksum of the attached `init.lua` is
16034358af0ef4a9a842834dd3a4c4db5181ee3a11fe9760ae0882507cc657c0

[Johan Sjöblom]

I will also post the SHA-256 checksum of my original init.lua. If your own installed file has exactly the same checksum, you have the same version that I tested and can replace it directly with the corrected init.lua from my previous post.

Original init.lua eligible for direct replacement:
36f523fe4e2ea9064c3cd1a9e5be9bb63e0caada6c2cabb3efa086f10d799314

Corrected init.lua:
16034358af0ef4a9a842834dd3a4c4db5181ee3a11fe9760ae0882507cc657c0

And the sha check command:
sha256sum /opt/picochess/engines/mame_emulation/plugins/chessengine/init.lua

-- Johan

[Dirk]

Johan,

I have not checked the code but keep in mind what I said before which was relevant for the chess lib in V3: From time to time the chess lib would send a POSITION FEN command (with move list) to the engine during a normal game instead of POSITION STARTPOS with move list (don't know the reason why - probably when move list gets too long!?)

if we in this case wouldn't process just the last move from the move list like before then ...

a) in case the mame engine supports position set up: your change would probably work but it would be way too slow when this happens during a normal game (as the positions et up for mame engine does this by entering manual set up mode and setting piece by piece to the correct square and we would have to set correct castling rights etc.

b) in case the mame engines does not support the position setup feature: we still would need the current behaviour by just accepting the last move and applying it to the current position

As I have not checked you probably might have considered this already...

Dirk  

[Johan Sjöblom]

Good point Dirk.

And yes, the above Lua fix takes this into account and supports both valid UCI forms:
position startpos moves ...
position fen <FEN> moves ...

the Lua correction improves MAME stability in both V3 and V4 because now it supports both format, whichever is sent by the chess library.

The new chess library does not switch to FEN because the move list becomes long. It uses startpos when the recorded game starts from the standard position and fen when the recorded game has another starting position.

I also tested it on my Pi 4 now and it works... But I dont get an initial "Set Pieces" message as I intended when you ask for Set Pos. Its very important however to wait for the "OK" before starting to play. It takes a few seconds to set up a retro engine.

[Dirk]

Sounds good - I will test soon …

To view this discussion visit https://groups.google.com/d/msgid/picochess/183b77e4-2890-4896-b8c3-490cb66972a5n%40googlegroups.com.

[Johan Sjöblom]

I dont know why MAME engines dont support takeback, but isnt this new setpos with a move stack (minus latest move) the same as a move takeback…?

[Randy Reade]

Not all retro machines allowed you to set up a position and setting up a position is not exactly the same as taking back moves in a game started from startpos.

[Randy Reade]

I think some retro machines support take back but it would need to be supported in the machine LUA file as well as init.lua. Dirk provided a workaround with his PicoTitor blunder check before sending the move to the engine.

[Dirk]

Yes, exactly Randy.

This is the reason: each machine has a different takeback behavior and to support all these different actions on the board of the Chess computer or via the keys what mean that we almost need for each machine, a different lua take back routine. 

And the other possibility to set up a position and then execute a move list is also not supported by very old machines.

Sometimes they can’t even set up a position sometimes they only can set up position, but don’t allow to enter multiple moves etc.

Set why we only have a blunder take back possibility for all machines which I have implemented in v3.

Am 21.08.2026 um 16:58 schrieb Randy Reade <randy...@gmail.com>:



--

You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/picochess/CAE5Wduucttd5Ev1sii_d0pkQ_ggs0tNoZ2s3JYyG-kwPx9ZhUA%40mail.gmail.com.

[Johan Sjöblom]

Thats clear! It sounds like that after we have stabilized the Set Pos and Scan, we might want to add a flag to the uci file that would prevent the move stack from being sent, but it might be easier to let a flag prevent Set Pos for certain mame engines. Just like we added the possibility to set analysis = false earlier...

But let's test first with the mames like mm V etc

[Johan Sjöblom]

I have tested a little bit and technically it seems to work. But I need to check the order of the voice announcements. Now I got an early "set pieces" but it comes so early that it is heard before the new position is set up. The sequence should be more "Set Pos" -> Please Wait -> New Position -> Set Pieces -> Board setup completes -> OK. Lets see if I can change the audio messages.

Technically it does seem to work. I am testing with Pi 4 and the mm V engine.

-- Johan

[Johan Sjöblom]

A new audio solution is now in master... Technically there should be no difference in handling the MAME engines, but the announcements are hopefully more of the sequence stated in the prevous post.

[Johan Sjöblom]

Yes, the "please wait" message is really needed. For a setpos with most of the pieces still on the eboard the setup takes 20 seconds for mm V. Thats the time between "Please wait" and "New position".

The 20 seconds is the time between isready asked from mm V, and when its ready. I can also hear that the first beep audio from mm comes after that. Technically this works, but you need to be patient and wait 20 seconds after the "please wait" message.

[RandyR]

I think there is still an issue. I need to do more testing but after the Set Pos, the MAME artwork does not match the selected position. It is showing the 'next move' as if it was played again.

Test setup: Pi5, Mephisto MM V, Millenium Chesslink eBoard

Mate in 3 (White to move):

Set up position and Scan. Switch sides to force engine to move as White. Hopefully it finds the mating sequence. Make the moves:

1. Bg7+  Kxg7  2. h8=Q+  Rxh8  3. Nh7#

In web UI move list, select Kxg7 to go back to that point. Use Set Pos. Web UI board resets. Reset pieces on eBoard to match web UI. Hear OK. But MAME artwork shows:

Randy

[Johan Sjöblom]

Good testing. And I found the root cause. The Lua will not make sure its the users turn to move so the engine played a move for white after Set Pos.

I was able to repeat it on my development machine using a different position than yours.

The fix is in the attached init.lua:

After replaying a move stack, the adapter now leaves MM V idle by setting my_color opposite to ply (move turn). That makes sure that the engine will not make the extra move. The position and actual side to move remain unchanged in the position. Only a later UCI go assigns my_color = ply and starts the emulated computer. I tested both a situation where it was my user turn, and when it was not. If its not my turn I need to use the lever switch-sides, but it works for me.

The new attached init.lua checksum is this:

1d04990a894e1b94d5c105bb1ea0e34150879dfef35cce5f29a97e949ebbf10b

-- Johan

[Johan Sjöblom]

This new Lua (also the previous Lua fix) actually makes the graceful recovery of an engine work better. In one of my tests the mm engine stalled and despite having time set to just some seconds it was thinking for more than 30 seconds. Maybe it would have responded eventually, but I tried to force a move, and the mm engine crashed. However, the graceful recovery worked exactly as planned and restarted the engine for me. And now because the engine was able to receive the starting position plus my existing move stack I was able to continue the game just by clicking the switch-sides... Nothing was lost.

So this lua fix also makes it possible to continue playing against a mame engine that can take a starting pos + move stack. The new lua at least sends it correctly. If every engine can do it, thats another thing then.

-- Johan

[Dirk]

The retro info function shows which features a mame engine supported (it is detected via a corresponding lua function existence check (I forgot the exact name of the position set up lua function) so we could check this before we are sending a position or position move stack to a mame engine…

Am 23.08.2026 um 10:24 schrieb Johan Sjöblom <messi...@gmail.com>:

This new Lua (also the previous Lua fix) actually makes the graceful recovery of an engine work better. In one of my tests the mm engine stalled and despite having time set to just some seconds it was thinking for more than 30 seconds. Maybe it would have responded eventually, but I tried to force a move, and the mm engine crashed. However, the graceful recovery worked exactly as planned and restarted the engine for me. And now because the engine was able to receive the starting position plus my existing move stack I was able to continue the game just by clicking the switch-sides... Nothing was lost.

--

You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/picochess/6ec5d0c1-865c-482f-be79-c3c827e8ce3cn%40googlegroups.com.

[Johan Sjöblom]

Good hint, thank you Dirk.

The important interface functions are:

• interface.set_pos: the emulated computer can accept a custom FEN/setup position.
• interface.edit_game: the emulated computer can replay a move stack.
• interface.show_info or interface.show_pv_after_move: analysis information is available.

Therefore the theoretical capabilities are:

• startpos + moves: requires interface.edit_game
• custom FEN without moves: requires interface.set_pos
• custom FEN + moves: requires both interface.set_pos and interface.edit_game

BUT: The current Picochess implementation only picks up "pos + info".... I need to add code to also pick up "edit"

• pos = interface.set_pos exists

• edit = interface.edit_game exists

• info = interface.show_info or interface.show_pv_after_move exists

If this works we can also replace the earlier extra uci parameter "analysis = false" that we had to put on at least two mame engines. If you remember when I implemented the support for getting engine info lines when the engine was thinking, ... Then we had to add analysis = false for some engines that could not produce that info. Also that information can be read from the lua adapter it seems. But that would be for a later fix. Now I will concentrate on trying t read the "edit" into Retro Info. If its not supported we should not send moves.

-- Johan

[Johan Sjöblom]

Issue opened for the above enhancement: https://github.com/JohanSjoblom/picochess/issues/449

I need to code it in a feature branch 449-detect-mame-capabilities

[Johan Sjöblom]

And yes Dirk, I was thinking the same about finding a way to include the init.lua script into the repo without bringing in all the engine specifics.

If the folder would always be the same it should be doable... and the mame folder in general is not depending on if its Linux or Pi.

[Johan Sjöblom]

The branch now contains a solution that does not send a move stack to a mame engine if it does not report the "edit" capability.

I tested Set Pos with Boris 1978 and mm V.

I opened a PR for this: https://github.com/JohanSjoblom/picochess/pull/450

Set Pos, Read Game, and engine recovery. Position-capable engines without edit instead use the current final FEN as a fresh root.

-- Johan

[RandyR]

I think something has interfered with the MAME artwork window switching. Lifting and replacing the King doesn't bring the artwork to the front or send it to the back. At least for me.

My setup: previously mentioned mate-in-3 position on board, MM V active engine when PicoChess starts up.

Initially, on a reboot (after updating RPiOS, updating PicoChess and using the latest init.lua) the MAME artwork opened behind the kiosk browser. I couldn't switch windows by lifting/replacing the King. I rebooted and the MAME artwork opened 'in front'. I tried to send it to the  back but no joy. alt+TAB on a keyboard works, as does 'WAYLAND_DISPLAY=wayland-0 XDG_RUNTIME_DIR=/run/user/1000 ydotool key 56:1 15:1 15:0 56:0' via SSH. So, it doesn't appear that the RPiOS update caused it. Debug log shows King lift is detected correctly.

Randy

[RandyR]

The Set Pos is working correctly now. Thanks, Johan.

[Johan Sjöblom]

The window switching should still work, try also with a non-King piece

The specification is still the same: With retro/MAME artwork active (rdisplay = True), lift any piece and replace it on its original square to switch between windows.

1. A quick lift-and-return switches windows—including the king.
2. If the king remains lifted until “set pieces” appears, returning it triggers PicoCoach/evaluation instead.
3. A non-king piece returned after “set pieces” still switches windows.

You could try this nr 3 above

It uses this, so check what window-control-backend you use. If you have auto, maybe try to force ydotool or swaymsg.

• X11: xdotool to send Alt+Tab

• Wayland: ydotool or swaymsg, selected through window-control-backend

• window-control-backend = none: disables runtime switching

[Johan Sjöblom]

I am pretty sure you need to set window-control-backend = ydotool

[Johan Sjöblom]

Oh and I have an unrelated question. What technology do you use to emulate windows chess engines? I dont think I ever tried configuring that. I kind of assume they are not mame/rom based, but maybe they are?

And if I want to study that, whats the best source? Just to have a look at the latest image setup or?

[RandyR]

Yes. That was it. I'm not sure why Auto stopped working for me.

[RandyR]

I use Box64 and Wine for x86_64 and/or Windows engines respectively. Easiest way to add them is to install Pi-Apps and then install Box64, then Wine (under Tools/Emulation).

[Johan Sjöblom]

Now the  449-detect-mame-capabilities branch is merged into master, so I would ask you to test that the Set Pos still works.

For me it works if I use mm V that advertises that it has "edit" capability. When I use Boris 1978 I lose the move stack because now Picochess listens to the "edit" capability and adapts.

This also means that set pos, engine recovery and load game works for Boris 1978 as it does not try to send a move stack to a mame engine that does not support it.

-- Johan

[Luigi Vogliobene]

Thank you Randy

[RandyR]

We seem to have lost the e.g. (pos + info) text which used to show beside the engine name when the retro engine is loaded.

Randy

[RandyR]

Also, I tested MM V and it is also working for me. But Fidelity Sensory Chess Challenger 9 doesn't switch sides properly (Scan function). FEN: 5k2/pp4p1/4pp2/1P6/7p/P3PKP1/5P1b/2B5 b - - 0 1

I think this is a problem with the fscc9.lua or maybe the engine itself. It doesn't respond correctly using the command line either (via SSH).

WAYLAND_DISPLAY=wayland-0 XDG_RUNTIME_DIR=/run/user/1000 ./mess -plugin chessengine fscc9
uci
id name Fidelity Sensory 9 (pos)
id year 1982
option name Speed type spin default 100 min 0 max 10000
option name Level type spin default 1 min 1 max 9
option name Clear announcements type check default true
uciok
setoption name Level value 3
position fen 5k2/pp4p1/4pp2/1P6/7p/P3PKP1/5P1b/2B5 b - - 0 1
go movetime 10000

... and it doesn't reply (but indicates a White king move, Kg2, without moving the king, on the MAME artwork. I suspect there may be a few retro engines that need lua fixes. It's a lot of work as Dirk mentioned, to maintain these when MAME core code updates. Fortunately, the retro chess engines have begun to stabilize in the MAME code.

I'll find some other retro engines to test.

Randy

[Johan Sjöblom]

Made 2 commits that at least temporarily brings back the MAME engine capabilities when its loaded. Should show on the clock display for a few seconds.

There is also a new "info"-tile in the engine menu... The ole clock retro info will also show the new "edit" capability, not only "pos" and "info".

That temporary display info should make it easier to test mame engines...

-- Johan

[Randy Reade]

Thanks, Johan. I'll find some time to do some more testing.