Testing help needed

[Johan Sjöblom]

DJ Dekker has made quality improvements to the uci communication. Together we have created a testing branch where we should make sure that everything still works. Basic testing works fine, and I also had AI test-play this branch against Mephisto, but the MCP interface is too slow to detect all error that could occur in uci communication timing.

If someone has a a possibility to test this branch it would be a big help. The branch name is: 492-508-integration. To test you need to have a terminal or ssh into the Pi and do:

cd /opt/picochess
git fetch origin
git switch 492-508-integration

It should behave like the latest master version.. DJ Dekker can comment if we would expect to see some behaviour changes.

[Andrea Gatti]

Hi, I can certainly test. Please explain what is the expected changes/behaviour I need to test/see.

Thanks.

AG

[DJ Dekker]

The branch 492-508-integration reworks the UCI engine communication under the hood. There are no new menus, features or visual changes. It should behave exactly like the latest master. If something looks different, hangs, or throws an odd message, that is what we'd like to hear about.

Where testing helps most (timing problems show up here):

• Switch mode, engine or level in quick succession, both during analysis and while the engine is thinking.
• Takeback: after takeback, a move the engine has already announced is now discarded and the game result is cleared (e.g., when the takeback move was a checkmate). Check that the board, clock and move list stay consistent.
• Web client clock and analysis: the local-language move notation and the live analysis updates were changed. Check that they look right.
• Variants (Three-check, Atomic, Racing Kings, etc.): the browser move history and FENs were changed. Check that the move list and board agree.
• If an engine stalls or restarts (e.g. MAME or a heavy engine), does it recover and does the game carry on afterwards?
• A full game on a real DGT board and clock with a time control, so the timing between board, clock and engine gets exercised.
• Remote engines have not been tested yet. If you have one configured, please try it.

When you encounter any problems, please post which engine, mode, board and clock you used, what you did, and what you expected versus what happened. Log excerpts are very welcome.

Greetings,

DJ

[Johan Sjöblom]

For testing I turned off the picotutor, I also gave the engine no_book. Especially turning off the tutor puts more stress on the engine analyser if you make rapid moves, because now the engine must keep switching between playing and analysis.

So far I have not found anything.

[Johan Sjöblom]

Testing with Codex did not reveal anything, but I switched to Windows and let Claude test it using the picochess mcp service. Here are the preliminary results of Claude stressing the engine switching (model Opus 5.5)

Hi,

I tested the 492-508-integration branch today on Windows (NOEBOARD, web client). The setup was Stockfish 19 at Elo@2200 with the Tutor off and the book set to nobook, so the engine had to switch between analysing and playing on every half-move. Time control 5+3. There is only one engine on this machine, so I switched levels instead of engines. The DGT board, remote engines and MAME are still to be tested.

To make the changes faster than clicking, I sent the requests straight to /channel, the same requests the web client sends, 0.3 seconds apart.

Problems found:

1. A level change while the engine is thinking loses the engine move and swaps sides.
   After my move 3.Nc3 I sent three level changes (Elo@1700, Elo@2500, Elo@2200) while the engine was thinking. The log shows:
     ERROR engine - go: go called but no engine loaded
   The engine never replied. PicoChess then treated me as Black: 3.Nc3 was credited to the engine, and the PGN header changed to White "NN" (Elo 3645), Black "Player". Switching sides back to White got the engine moving again (3...Bb4) and corrected the headers.
2. Fast level changes leave orphaned engine processes.
   Four level changes 0.3 s apart on my turn left two extra a-stockf.exe processes running idle. The burst while the engine was thinking (problem 1) left two more. The engine that remains works normally. The log shows:
     WARNING engine - send: engine terminated while configuring options []: engine process died unexpectedly (exit code: 1)
     WARNING engine - start_analysis: start analysis requested but no engine loaded
   With about 3 seconds between level changes, no processes were left behind.
3. A fast mode change while the engine is thinking is lost.
   After my move 4.e5 I sent Ponder, Kibitz and Normal 0.3 s apart while the engine was thinking. PicoChess ended in Kibitz instead of Normal. The log shows:
     INFO mainloop - think: skipping superseded engine search revision 7
   The same kind of burst on my own turn (Ponder, Kibitz, Analysis, Normal) correctly ended in Normal with the position unchanged.
4. No engine analysis on my turn after the first engine move following a cold start.
   After a cold start of PicoChess I played 1.e4 and the engine replied 1...e6. On my turn the engine used no CPU, and the only analysis published was the engine's own search of the previous position (Black to move). A level change started analysis, and after that it started normally after every engine move. In a New Game later it also worked from the first move, so it may be limited to the first search after startup. I have not tried to reproduce it with another cold start yet.
5. A warning that appears often:
     WARNING engine - stop_async: engine analyser stop timed out after 1.0s - cancelling task
   It was logged 7 times in about 10 minutes, mostly right after my moves. I saw no visible effect.

What worked:
- Normal takeback in the middle of the game (my move plus the engine reply): the board, move list and analysis were all consistent.
- Takeback while the engine was thinking: the search was discarded and no late engine move appeared.
- Takeback of a checkmate (2.g4 Qh4#): the result went from 0-1 back to *, and the position and move list were correct. Taking back only the mating move also worked: I continued as Black and the engine played on as White.
- Level changes about 3 seconds apart, both on my turn and in a game in progress.

Not tested yet: the web client clock and analysis notation, variants, the DGT board with a clock, remote engines, and MAME stall/restart.

[Johan Sjöblom]

I told Claude that these "stress" tests are not likely scenarios when a human is using the program... Then I got some new test results as below. I will now switch to test the same in master so that we would know what is actually a regression and what is not. Here are the greetings from Mr Claude on his more realistic tests:

The realistic tests change the picture: the most important bug doesn't need fast clicking at all.

1. A single level change while the engine thinks loses its move and swaps sides. This is realistic.
- It happened 3 times out of 3, with the change sent 0.5 s, 2 s and 4 s into the engine's think.
- Each time PicoChess made you the side to move and the engine never replied. The first time the log showed go called but no engine loaded.
- On both master and this branch, a level change mid-think deliberately forces the engine's move and throws it away (get_rid_of_engine_move), and nothing restarts the search afterwards. So this may be old behaviour that the branch kept. Only running master would tell us whether it's a regression.
- The PGN header wasn't updated either: the engine was at 2000 while the header still said BlackElo "2200".

2. Orphaned engine processes only come from overlapping changes. Single level changes left nothing behind. The four orphans still running are all from the 0.3 s bursts. That makes it a stress-only problem, lower priority.

3. Returning from Ponder when the engine was mid-think leaves the engine stalled. This is new, and it happened 2 times out of 2.
- I switched to Ponder 3 s into the engine's think, then returned to play.
- PicoChess restored the position with you as White and Black to move, but the engine never started its search. It only ran background analysis (depth 33 and climbing), and the game stood still until I pressed Move now.
- The return goes through set_wait_state(preserve_play_mode=True), which waits for the user. The branch only re-indented that code, so this most likely comes from the PONDER checkpoint feature rather than the UCI changes. It's probably one for you rather than the branch author.

4. A Kibitz round trip mid-think is fine. PicoChess made you the side to move when returning to Normal, which is the usual rule, so nothing stalls.

5. engine analyser stop timed out after 1.0s appears after nearly every move you make. It came 8 times in this round. Nothing visibly breaks, but stopping Stockfish should take milliseconds, so something in the stop path waits too long.

[DJ Dekker]

Hi Johan,

Thanks for your findings. I'm looking into these issues to see whether they are regressions.

I could reproduce problem 1 (a level change while the engine is thinking swaps sides) on master. A level change from the web client is sent as two events, EVT_LEVEL followed by EVT_NEW_ENGINE. The NEW_ENGINE handler calls get_rid_of_engine_move(), which forces the engine to move and then deliberately discards that move (ignored engine move - takeback or state change forced move). It then reloads the engine and calls set_wait_state(msg, ...) without preserve_play_mode. That re-derives the play mode from the side to move, so after 1.a4 the user suddenly plays Black (new play mode: PlayMode.USER_BLACK), and no new search is started.

I'll keep investigating...

Greetings,

DJ

[Johan Sjöblom]

Good news. The above posts are problems mostly in master already... I am still testing... Next post will have better conclusions. Looks like good news for this test branch 492-508-integration :-)

[Johan Sjöblom]

Here are the summary greetings from Mr Claude (Opus 5.5.) after he has tested both the branch and the master to understand what it regression and what is not. Notice the sentence "uci communication held up better on the branch". A nice success DJ Dekker :-)

I tested the 492-508-integration branch on Windows today (NOEBOARD, web client). The setup was Stockfish 19 at Elo@2000-2200 with the Tutor off and the book set to nobook, so the engine had to switch between analysing and playing on every half-move. Most tests used 8 seconds per move to get a predictable thinking window. I ran every test that showed a problem on master as well, so I could separate regressions from existing behaviour.

To make the changes faster than clicking, I sent the requests straight to /channel, the same requests the web client sends, 0.3 seconds apart. That is a stress test rather than normal use, but on a Pi or with slower engines the same overlap could happen at human speed.

Summary: the UCI communication held up better on the branch than on master. I found one regression in fast mode switching.

REGRESSION: a fast mode change while the engine is thinking is lost

Steps:
1. Normal mode, user plays White, engine thinking about its move.
2. About 0.8 s after the user move, select Ponder, then Kibitz, then Normal, 0.3 s apart.

Branch: PicoChess ends in Kibitz instead of Normal. This happened in 4 out of 4 runs. No UCI errors are logged. The log shows:
  INFO mainloop - think: skipping superseded engine search revision N
  INFO mainloop - _save_position_checkpoint: temporary analysis checkpoint saved ...
There is no "mode cant be changed" warning, so the Normal request is not rejected by that guard. It seems to be dropped or handled out of order.

Master: PicoChess correctly ends in Normal (2 out of 2 runs).

The same kind of burst on the user's own turn (Ponder, Kibitz, Analysis, Normal) ends correctly in Normal on the branch.

My guess is that it is related to the reworked start of the SET_INTERACTION_MODE handler in 7e939b60 "Fix mode switch race with engine move", but I have not confirmed that.

IMPROVEMENT: no UCI protocol failure under the same burst

On master, the burst above broke the engine protocol in both runs:
  ERROR base_events - exception while parsing engine output ... asyncio.exceptions.InvalidStateError: invalid state
  WARNING engine - continuous analysis protocol failure: AssertionError: CommandState.NEW - restarting engine Stockfish 19
  ERROR base_events - UciPingCommand failed after returning preliminary result
On the branch the same burst produced no protocol errors at all.

WORKED on the branch:
- Takeback in the middle of the game: the board, move list and analysis were consistent.

- Takeback while the engine was thinking: the search was discarded and no late engine move appeared.

- Takeback of a checkmate (2.g4 Qh4#): the result went from 0-1 back to * and the position was correct. Taking back only the mating move also worked, and the game continued.
- Level changes a few seconds apart on the user's turn.
- Mode switches on the user's turn, even 0.3 s apart.

SAME ON MASTER AND THE BRANCH (not caused by the branch; I will open separate issues):
- After a cold start, with the Tutor off, the engine does not analyse on the user's turn until it has been restarted once, for example by a level change.
- A level change while the engine is thinking discards the engine's move (get_rid_of_engine_move) and does not restart the search. PicoChess then gives the user the side to move, and the PGN header keeps the old Elo. This happens with a single change, not only with fast changes. The log shows "go called but no engine loaded".
- Returning from Ponder (the checkpoint return) after Ponder was entered while the engine was thinking: the engine never resumes its search, and only "Move now" gets it going.
- Four level changes 0.3 s apart leave two orphaned engine processes running.
- "engine analyser stop timed out after 1.0s - cancelling task" is logged after most user moves.

Not tested yet: the web client clock and analysis notation, variants, a real DGT board with a clock, remote engines, and MAME stall/restart.

[Johan Sjöblom]

Some bookkeeping: Follow-up on the potential bug list above: I opened GitHub issues in master for the two bugs that are the same on master and on the branch, they are not caused by 492-508-integration:

#522 Level change while the engine is thinking discards its move and gives the user the engine's side
https://github.com/JohanSjoblom/picochess/issues/522

#523 Returning from temporary analysis (PONDER) leaves the engine stalled if PONDER was entered while it was thinking
https://github.com/JohanSjoblom/picochess/issues/523

#522 is in the engine-switch path (get_rid_of_engine_move), close to what your branch touches. If it fits naturally into your work, feel free to take it. Otherwise I can look at it after this branch is merged.

I have not opened issues yet for the missing analysis after a cold start or the orphaned engine processes, since those are in the engine lifecycle your branch is changing. I will check them again once your work is finished.

[Johan Sjöblom]

The above two issues 522 and 523 are now fixed in master, and I tested them by running noeboard. If we want to rebase these into this non-merging integration branch there is a small merge conflict fix that is needed.

--> I will wait for your comment DJ Dekker on how you want to proceed. You can rebase it, or I can rebase it.

After rebase after we test the integration branch we don't need to worry about the two bugs found and fixed in master. Its 2 commits in master that are not in this integration branch. A fairly mechanical merge conflict according to our friend Claude.

// Johan

[DJ Dekker]

Thanks, Johan. Please go ahead and rebase 492-508-integration onto the latest master and resolve the small conflict. That makes sense, as it will ensure that further testing of the integration branch also includes the fixes for #522 and #523.

Please let me know when the rebased branch is ready, and I will update and test it again.

Greetings,

DJ

[Johan Sjöblom]

Now it is rebased :-)

[Andrea Gatti]

Wow I said I would test, but I cannot cope with your speed guys! :-)

[Johan Sjöblom]

Thanks! We have just made some basic tests and fixed 2 issues so that testing can continue.

If you already had the branch you need to do git pull to get the two fixes.

I still have much testing to do. And also test with the DGT board. At the moment this test branch feels better than master, but we need to test more still…

DJs first post in this thread is still a valid starting point for testing

[Johan Sjöblom]

Another run with "Claude the tester" on rebase 492-508-integration, (with the fixes for #522 and #523)

I tested it again today on Windows (NOEBOARD, Stockfish 19, Tutor off, nobook, 8 seconds per move). I will open more issues on the master. Here are Claudes findings:

Good news first:
- No UCI protocol errors in the whole test round.
- The #522 fix still works next to your engine changes: a level change 0.5 s after the user move logs "go called but no engine loaded", followed by "engine search revision N was not started", and the search task ends instead of waiting forever.
- The #523 fix still works: switching to PONDER while the engine thinks and then returning makes the engine resume its search by itself.

The lost mode change from my earlier report is still there: Ponder, Kibitz, Normal sent 0.3 s apart while the engine thinks ends in Kibitz. I think I found the root cause, and it is older than #518.

event_consumer() in mainloop.py does not process events one after another. It starts every event as its own task, 0.05 s apart (the "issue #45" comment). So the three SET_INTERACTION_MODE handlers run at the same time and interleave at every await, and whichever handler sets interaction_mode last wins. The switch to Normal finishes fastest because a switch to a playing mode skips get_rid_of_engine_move() and its 0.5 s sleep, so the slower Kibitz handler overwrites it. #518 moved some awaits and so changed who wins the race. Master gets the right result by luck, not by design.

Suggestion: an asyncio.Lock around the SET_INTERACTION_MODE handler. asyncio.Lock serves waiters in order, so the mode changes would be applied in the order they arrive.

The orphaned engine processes have the same cause. Four level changes 0.3 s apart again left two idle engine processes. The NEW_ENGINE handler has no lock either: overlapping handlers each quit self.engine and assign a new UciEngine, and an engine that is opened but overwritten before anyone quits it keeps running. The same kind of lock around NEW_ENGINE would prevent that.

One more observation, also on master: "engine analyser stop timed out after 1.0s - cancelling task" is logged after most user moves. The engine's clock is already running while that stop completes, so each timeout costs the engine about one second of its own thinking time. From the code I would expect the stop to take milliseconds, so something waits longer than it should. I have not traced it yet; debug logging on the engine module for one run should show where the second goes.

Finally, the missing analysis on the user's turn after a cold start is not caused by your PRs. It comes from my own earlier change that only allows engine analysis after set_mode() has been called, which the cold-start sequence never does. Thats a master bug issue.

[Johan Sjöblom]

Issue opened on master: https://github.com/JohanSjoblom/picochess/issues/528

I will fix it when I have time.

[Johan Sjöblom]

And an investigation is needed on https://github.com/JohanSjoblom/picochess/issues/529

so opened that one too. Engine looses 1 second thinking time in some situations on the clock

[Johan Sjöblom]

Fixed issue #528 listed above and rebased the integration branch to include the fix.

[Johan Sjöblom]

Fixed issue #529 as well, but it stays as a PR until we have tested the branch 529-stop-engine-analysis on a physical DGT Pi 3000 clock. So its not rebased into this integration test branch.

If the fix works it fixes the lost 1 second on engine clock. The DGT clock timing is sensitive and I have had problems with exactly this section before (which is why there was an async wait there causing this). Now we have a candidate that makes the clock even more accurate.

[Johan Sjöblom]

OK, fixed some more bugs in master and rebased the integration branch 492-508-integration that we want to test neatly on top of master.

- Correction to previous post: issue #529 with PR #531 needs to be tested on a DGT clock before we merge it to master.

The open question for you DJ Dekker is that I have fixed issue #527 in PR #532. Its the eval showing after checkmate fix. I could merge that one into the integration branch 492-508-integration. But it depends if you want to fix it in 492?  Let me know. If you want me to merge into the integration test branch I will do my best to solve the possible conflict.

Most issues are now fixed... Testing of the integration branch remains... And the special 529 on a DGT clock.

[DJ Dekker]

Hi Johan,

Great, thank you! I'll do more testing in the weekend.

Yes, please merge #532 into the integration branch rather than fixing #527 in #492. I’d like to keep #492 focused. The conflict is mainly in the mode-transition clock handling in app.js: #492 clears stale clock text when transitioning into or out of Ponder/free-analysis, while #532 preserves the game-end result when entering that mode. Please preserve both behaviors: clear stale text during normal transitions, but keep the result text for a finished position. Could you please also combine the tests from both versions of test_web_analysis_clock.py?

Greetings,

DJ

[Johan Sjöblom]

Done :-)

PR #532 is now merged into 492-508-integration. I resolved the clock display conflict and combined the test_web_analysis_clock.py tests from both branches. The relevant tests pass. In addition issue #505 was also fixed in master, and the integration branch includes that fix after the rebase. The integration branch is pushed and ready for testing.

This means that more or less all bugs in the issue list are now in the integration branch for testing. The issue #529 with PR #531 I still keep as a separate branch and PR as it needs testing on a DGT clock. Its not yet included in either master nor the integration branch. All others should be.

Happy testing :-)

This is a good hardening sprint. Lets try to avoid adding new features while we are hardening. Refactoring is ok of course, but lets try to keep a feature stop for a while? It would be good to get this integration branch and all the bugs fixed stabilized before continuing.

// Johan

[DJ Dekker]

Hi Johan,

I tested PR #531 at commit 2ab8bf2c on my RPi2 with a DGT3000 clock and the web client.

1. Normal game with a 5+3 time control: passed.
2. Engine analysis during play, checked through the web client because analysis is not shown on the DGT3000 in Play mode: passed.
3. Engine/level switching and short fixed-time play: passed.
4. Fixed 5 seconds per move: the DGT3000 often adds one second just before displaying the engine move. An A/B test on #531’s direct predecessor (58477824) showed exactly the same behaviour, so this is an existing DGT clock-stop/display bug, not a regression caused by #531.
5. Rapid analysis/search transitions and starting a new game while the engine was thinking: passed. No hangs, stale engine moves, analyser timeouts, forced stops, or overlapping searches occurred. The same pre-existing extra-second display issue as in test 4 was visible here as well.

Conclusion: tests 1, 2, 3 and 5 passed. Test 4 exposed a pre-existing DGT3000 issue, but I found no regression or remaining blocker for PR #531.

Greetings,

DJ

[Randy Reade]

I think that extra second was to compensate for the time between when the engine actually responded and the clock displayed it. I could be wrong.

Randy

--
You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/picochess/604f45dc-971d-4dc4-a805-6014e55a3940n%40googlegroups.com.

[Johan Sjöblom]

Nice testing. To me it means that I can add the PR #531 to master and to the integration test branch.

[Johan Sjöblom]

The PR #531is merged into master and also rebased into our testing integration branch. Now we can test the 492-508-integration branch as it sits nicely on top of master.

For the number 4 there is a new issue opened: https://github.com/JohanSjoblom/picochess/issues/536. Stopping the DGT3000 with the last reported time is intentional. Making the displayed time jump forward by one second does not appear to be. There is no explicit one-second addition in that code path flow. The jump could be a timing side effect if the cached report is older than the clock’s current display. That is still a hypothesis; the logs and a physical clock test will tell us whether it explains the observation. There is some indication on what it could be in the issue opened, but it needs further investigation.

BR, Johan

[Johan Sjöblom]

Hi DJ. Some good test results from my side. One thing that was noticed as a side issue is that in very quick mode changes (1.2 second or less because of fixed waits on 0,5+0,7) the web client gets confused about the mode. It might help that the users need to be aware of that doing mode changes you have to be patient. If you wait for "OK Mode" there is no problem. I opened issue #537 if we later want to improve the UX of mode changes. I don't consider it urgent.

The test we would need is from a slow uci engine, maybe MAME? That would really tell us if the uci communication deals with timeouts.

Here are the greetings from tester Claude:

I tested the rebased 492-508-integration branch again today (14 commits on top of current master, which now also contains the fixes for #520, #522, #523, #526, #528 and #529). Windows, NOEBOARD, Stockfish 19 at Elo@2200, Tutor off, nobook, mostly 8 seconds per move, driven through the picochess MCP.

Summary: the UCI communication held up in every realistic scenario. There were no protocol errors in the whole test round.

What I tested, all clean:
- Normal play after a cold start: the engine analyses on the user's turn from the first move, and the replies come on time (9.5-9.7 s after the user move at 8 s per move).
- Level change on the user's turn.
- Level change 0.5 s after the user move (before the search starts): "go called but no engine loaded" is followed by "engine search revision N was not started", so the search task ends cleanly and the user gets the side to move as designed.
- Level change 4 s into a running search: stopped cleanly, one engine process left.
- Takeback, Move now and New Game while the engine is thinking: no late engine moves, no warnings.
- Entering PONDER while the engine thinks and returning: the engine resumes its search by itself.
- Analysis mode: moves for both sides, three engine lines, takeback and hint.
- 5+3 game: no warnings.

Two small observations:

1. Once, right after entering Analysis mode from a fresh new game, the analysis read through a new websocket connection (the replay a new client gets) showed the previous position's lines in UCI notation under the new position's FEN. The live analysis stream to an open client was correct, and I could not reproduce it in two retries. I mention it in case it rings a bell.
2. "engine analyser stop timed out after 1.0s" still appears occasionally on other stop paths (for example during mode switches). It is harmless, just noise in the log.

As a stress test only, I also sent mode changes about 0.3 s apart while the engine was thinking. That is not a realistic user scenario, so it is not a priority for these PRs. I wrote it up separately as #537, with a workaround and a possible fix: https://github.com/JohanSjoblom/picochess/issues/537

Not tested: a real non-responding ping/readyok (Stockfish always answers), the DGT board, MAME and remote engines.

From my side the branch looks good for the realistic cases.

[DJ Dekker]

Hi Johan, I have now completed an additional hardware test round of 492-508-integration at commit 5e79c677.

1. General integration results

• RPi5 with Touch Display 2, without DGT3000/DGTPi: normal play and the web client worked correctly and remained synchronized.
• RPi2 with a physical DGT board and DGT3000: normal play also worked without general stability problems.
• The previously reported stale analysis after opening a new websocket could not be reproduced in several attempts using Brave and a newly opened Firefox client. Both clients showed the same FEN, moves and analysis.

2. Fast mode switching — issue #537

Normal mode changes work when sufficient time is allowed. Rapid web mode changes still expose the known timing problem:

• each web set_mode request can take approximately 2.45–2.50 seconds;
• event-loop timers become about two seconds late;
• Texel was restarted several times because continuous analysis was cancelled while a readiness ping was pending;
• one restart produced a BrokenPipeError, although PicoChess recovered.

“Wait for OK Mode” is only a useful workaround on a physical DGT clock. The web client provides no equivalent completion indication. Web users currently need to wait about three seconds between mode changes.

3. Analysis → Play display race — issue #540

This is reproducible on the integration branch after all, although not on every attempt.

After clicking Return to Play:

• the checkpoint is restored;
• the backend returns to Mode.NORMAL;
• the normal clock briefly appears;
• an analysis display message then overwrites the clock again.

The log shows:

• 20:56:12.386: return to Mode.NORMAL;
• 20:56:13.675: DGT_DISPLAY_TIME restores the clock;
• 20:56:14.939: analysis overwrites it with Ply 8 | score 5.37 | Dd3;
• 20:56:23.175: another update appears as Ply 9 | score 9.98 | Dd3.

Further DGT_DISPLAY_TEXT commands continue every second while PicoChess is already in Play. The web client therefore shows an analysis line where the clock should remain visible. The DGT3000 also receives the corresponding score-only text. modeswitch5.log contains the reproduction.

4. DGT3000 score display — issue #539

When the unwanted analysis display reaches the DGT3000, it can show bare numerical scores rather than the normal clock. This may share its underlying analysis-routing cause with #540.

5. Dutch menu translations — issue #538

More untranslated menu content was found. The Mode submenu is dynamically generated and most buttons remain in English, although Dutch translations already exist. “Terug naar Spelen” is translated because that button uses the translation catalog, while the other generated tiles use their English labels directly.

6. DGT3000 extra-second behaviour — issue #536

The extra displayed second was reproduced both with PR #531 and on its predecessor/current master. It therefore does not appear to be a regression caused by #531. Issue #536 remains the correct place for further investigation.

Overall

Normal play on both the RPi5 and RPi2 looks good, and ordinary mode changes generally work. However, #540 is not obsolete: the integration branch still has an intermittent Analysis → Play display race. I would like your opinion on whether #540 should be fixed before merging the integration branch. The other findings are either already tracked separately or are non-blocking UX/localization issues.

Greetings,

DJ

[DJ Dekker]

Hi Johan, I also ran an additional mode-switch test on the RPi5 with Touch Display 2, without a DGT3000 connected.

I repeatedly switched between Play and Analysis, including fairly quick consecutive switches. I found no visible problems:

• Play mode consistently showed one PV;
• Analysis mode consistently showed three PVs;
• returning to Play mode restored the clock correctly;
• the mode remained stable;
• the web client stayed responsive and synchronized.

I checked two RPi5 logs. One contained three engine analyser stop timed out after 1.0s - cancelling task warnings, but these caused no visible malfunction and PicoChess continued normally. The second test log contained no engine, protocol or mode-switch errors at all; only unrelated force_leds_off: (rev) leds still on warnings.

This is useful as a comparison with issue #540: I have not reproduced that display problem on the RPi5. So far, it has only occurred during the RPi2 test with a physical DGT board and DGT3000 attached. I cannot yet distinguish whether that difference is caused by timing, load, or the physical DGT clock path.

Greetings,

DJ

[Johan Sjöblom]

I fixed a one-liner code change that affected both #539 and #540, but it does not fix them, it only makes trouble shooting on them clearer as it was disturbing them both. I created issue #541 for that. That fix is now both in master and in the integration branch 492-508-integration, so the one-liner issue is fixed and will hopefully help us to find some more information on issues #539 and #540.

Your test on your RPi2 and DGT 3000 clock is really valuable. Especially if the logs tell us something. Please pull in the new one-liner fix commit before you continue testing.

So yes, I think we could spend a little bit time on testing #540 on the integration branch. The clock line with Ply 8, score etc should now be gone as of my fix, please test :-) ... The 3 PV analysis lines needs investigation.

[Johan Sjöblom]

I think I have a potential fix for the issue #540. I created PR https://github.com/JohanSjoblom/picochess/pull/543 which merged the fix into the integration branch.

So hopefully the #540 is now fixed in the integration branch. The new commit in the integration branch is 4429f6e

[Johan Sjöblom]

Rock n Roll. I think I fixed also #539.

The fix for #539 in its own branch (539-dgt3000-eval-after-switch-to-analysis-mode), and I cherry-picked that single commit into 492-508-integration for testing.

The integration branch now has fixes for both #539 and #540, but neither of them is in master yet.

Could you test with your RPi2 and DGT3000? Hopefully both are solved.

[Johan Sjöblom]

I propose that the strategy is that we make a new PR to integrate the integration branch 492-508-integration into master. 

The main reason for this is that the merge conflicts are already solved there, and it does contain most of the PRs open, and also the latest issue fixes without any PR open.

Compared to yesterday #541 went into master as it made it easier for me to fix #540 and #539 which are as said, not in master. Those 2 are only in the integration branch for you to test.

If you agree, then we merge it into master, and we can continue regression testing there... And start to clean the PRs and issues that are fixed in master then.

BR, Johan

[DJ Dekker]

Hi Johan,

That's great news! I agree with your proposed strategy. Merging 492-508-integration as one integration PR into master makes sense, because the conflicts have already been resolved there and the branch contains the combined fixes and tested behavior.

My latest RPi2 test (without DGT3000, using only the web client) on commit 9365b43c confirms that the #540 MultiPV fix works: Analysis shows three PVs, Play returns to one PV, and the stale extra lines do not reappear. I also did not reproduce the earlier frozen Arasan analysis or illegal uci error.

One related issue remains: after Analysis → Play, the web clock is not restored immediately; it appears after the next move. I documented the timestamps in #540. I do not currently consider this a blocker for merging the integration branch, but it should remain as a separate follow-up.

Before giving the final hardware sign-off, I would like to run one short test on the same commit with the DGT3000 connected. The latest #539 readable-score fix and the physical-clock side of #540 have not yet been tested by me on 9365b43c.

If that focused DGT3000 test is clean, I support merging the integration branch into master and continuing regression testing there.

Greetings,

DJ

[DJ Dekker]

Hi Johan, I reproduced one remaining part of #540 on the RPi2 with the DGT3000, using integration commit 9365b43c.

The backend switches correctly from Ponder/Analysis to Normal, but the physical clock display is not restored immediately. During the slow analyser shutdown, the once-per-second DGT callback can still emit analysis text using the old display mode. After the mode change completes, no DGT_DISPLAY_TIME is sent. The DGT3000 therefore keeps showing depth/score until the next move triggers MSG_EXIT_MENU.

I propose a small follow-up to PR #543:

• synchronize dgtmenu with every accepted mode change before awaiting the potentially slow analyser shutdown;
• leave rejected mode changes and the existing PGN replay recovery unchanged;
• after an accepted transition into a playing mode, emit the existing Message.EXIT_MENU() so DgtDisplay restores either the clock or a genuinely pending engine move;
• add a deterministic regression test with a deliberately suspended analyser stop, plus checks for rejected transitions and normal Play analysis.

This stays within the existing async and display architecture: no delay, no direct DGT command from mainloop.py, and no suppression of legitimate analysis output in Play.

If you agree, I suggest adding this to PR #543 because it completes its #540 fix. If you prefer to keep #543 as-is, I can make it a separate focused PR against the integration branch.

Greetings,

DJ

[DJ Dekker]

A quick update on #540. Since #543 has already been merged, I did not add to it. The remaining DGT display problem is now a separate follow-up: #544 Restore DGT display after leaving analysis mode, targeting 492-508-integration.

What I found on the RPi2 with the DGT3000 (integration commit 9365b43c): the backend switched correctly from Ponder/Analysis to Normal, but the clock kept showing depth/score until the next move. During the slow analyser shutdown, the once-per-second DGT callback still ran with the old Ponder display policy, and no DGT_DISPLAY_TIME was sent when the switch completed.

The fix is small:

• dgtmenu is synchronised with every accepted mode change before the potentially slow analyser stop.
• After an accepted switch into a playing mode, the existing Message.EXIT_MENU() restores the clock, or a genuinely pending engine move.
• Rejected switches and the PGN replay recovery are unchanged, and legitimate analysis output in Play is not suppressed.

The tests include a deterministic case with a deliberately suspended analyser stop. pylama and unittest are green; mypy is red on the existing baseline, with the count unchanged at 256 errors.

[Johan Sjöblom]

Nice progress again :-)

I can take a look at PR #544 and its 2 commits... most likely they can be merged (to this integration branch).

[Johan Sjöblom]

Hi DJ,

I reviewed and merged both commits into 492-508-integration.

Moving the DGT menu sync before the slow analyser stop closes exactly the window you saw on the RPi2, and reusing EXIT_MENU to restore the clock (or a pending engine move) is a nice small solution. The tests cover the suspended-stop case well, and the full suite is green here.

Two small remarks:
- With overlapping mode changes (#537), the menu now follows the most recently started switch rather than the last one to finish. That is fine for now; just something to keep in mind if #537 ever gets its lock.
- Could you confirm on the RPi2 with the DGT3000 that the clock now comes back right after Analysis → Play?

Greetings,
Johan

[DJ Dekker]

Confirmed: on RPi2 with DGT3000 clock, the time display comes back right after switching Analysis -> Play.

[Johan Sjöblom]

We are close to merging the integration entire test branch to master.

I asked Claude what still might be open? Here is its list. the exact display on #539 could be a subjective matter, you decide if you want to change anything or if this is fine.

What's still open on the integration branch:
- #540, the three web PVs after Analysis → Play: fixed by #543 and tested here on the PC, but not yet confirmed by DJ on the RPi2.
- #539, the DGT3000 score format: the fix is on the integration branch; DJ still needs to check how " 4  0.42" reads on the real DGT3000.
- The late bestmove from a cancelled search (the Arasan "illegal uci" in Analysis mode): waiting for DJ's retest with a debug log, if it shows up again.

[Johan Sjöblom]

The PR #545 is now open waiting for our final decision.

[DJ Dekker]

What's still open on the integration branch:
- #540, the three web PVs after Analysis → Play: fixed by #543 and tested here on the PC, but not yet confirmed by DJ on the RPi2.

Confirmed on RPi2 with DGT3000: the web client shows three engine pv's when in Analysis mode, back to one pv after switching to Play mode. This is working correctly.

 

- #539, the DGT3000 score format: the fix is on the integration branch; DJ still needs to check how " 4  0.42" reads on the real DGT3000.

On the DGT3000 analysis output is still showing depth and score (like "6 -3.25") but not the pv. This is still not working correctly, but I think it's not blocking for the integration branch. I'll investigate further into this.

 

- The late bestmove from a cancelled search (the Arasan "illegal uci" in Analysis mode): waiting for DJ's retest with a debug log, if it shows up again.

I haven't found anything in the latest debug logs pointing at this anymore.

[Johan Sjöblom]

OK, so we only have the considerations around #539 left, although they do not block. Not sure what would be a correct output on the DGT 3000 clock.
Feel free to make commits to this integration branch. Of course you can go via a PR but as we are close to the end, the final touches could just be commits on this branch?

[DJ Dekker]

Hi Johan,

I checked the expected behaviour on both setups:

• On DGTPi (RPi3 via i2c to a DGT3000), Analysis correctly alternates between the PV and depth/score.
• On the standalone DGT3000 with RPi2, the readable depth/score from #539 worked, but the PV rotation was missing.

Here, “Analysis” refers to the user-facing Analysis mode, implemented internally as Mode.PONDER. The separate internal Mode.ANALYSIS is shown as “Move Hint” in the UI.

I have restored the same intended Analysis behaviour for the standalone DGT3000: alternating PV and depth/score according to the ponder interval. Kibitz remains depth/score only, and the web analysis display is unaffected.

The fix is based on the integration head that already contains #544, so the display mode is synchronized before a potentially slow analyser stop. I also confirmed on the standalone DGT3000 that returning from Analysis to Normal restores the normal clock display without a late analysis rotation.

The fix has been successfully tested on the RPi2 with the standalone DGT3000 and is ready in PR #546, targeting 492-508-integration.

With the readable score formatting already on the integration branch and the restored PV rotation in #546, I consider #539 fully resolved once this PR is merged.

Greetings,

DJ

[Johan Sjöblom]

OK, that commit is now also merged to our integration branch. I will make some minor checks but I think we are ready to merge the integration branch to master.

The PR is #545

Are you ok with merging it to master if I dont find anything extra in my small checks?

[DJ Dekker]

Yes, absolutely! 👍

Op zondag 25 oktober 2026 om 17:53:06 UTC+1 schreef messi...@gmail.com:

[Johan Sjöblom]

Done. Lets celebrate. I have a good Italian red wine here. Thanks for all the help everyone.d

Now we need to basic test master... and see if we missed something.

So to other testers: Now you can update picochess to the latest version and test there.

[Johan Sjöblom]

Claude basic testing found a new small thing... We probably should clean out all the issues that have been closed, but here is potentially a new one to be opened. As you see on my bolded text this is visible on a DGT clock?

New finding: the clock rotation after a checkmate in Analysis.
- After "Checkmate", the backend's clock text goes back to the previous position's evaluation, "depth 40 | mate in 1", every few seconds, and stays there. It happens with both the engine's own mate (Ra8#) and a different one (Rb8#), so it isn't the ponder-hit path.
- The web client isn't affected: the #527 logic keeps showing "Checkmate". But a DGT clock would show this rotation, standalone DGT3000 or DGTPi.
- It's become more visible with this merge: #541, #544 and #546 make the Analysis rotation run whenever Analysis is active, however it was entered.
- No analysis messages arrive after the mate, and the display's game-end handling had already run, since its hint is reset. So something sets the display's stored score back to "mate in 1" after the game end. Finding exactly what needs one run with log-level = debug. It would make a good new issue, or a question for DJ, who can see it on his DGT3000.

I opened a new issue for this one: #547  

I think I will use AI help to list all the issues and PRs  that we now can close after this succesful integration branch testing :-)

[Johan Sjöblom]

I closed the issues and PRs that we have fixed. If I made a mistake, just reopen them.

The open ones seem to be valid.

Lets take a moment to celebrate this nice cooperation. Thanks DJ Dekker for all the testing and fixing :-)

[DJ Dekker]

Hi Johan,

Wow, we really made some nice progress here. Thanks for all your patient work!

I'll have a look into #547 this week.

In the meantime I have started cleaning up the current mypy baseline on branch codex/typing/mypy-cleanup. Just want to get rid of all those mypy error messages.

So far, the number of errors has dropped from 256 in 18 files to 234 in 9 files. All changes are split into small commits and tested. Mypy also uncovered two real issues: an unawaited update notification and an outdated pyserial inWaiting() call.

The work is still in progress. You can follow it here:
https://github.com/djdekker/picochess/tree/codex/typing/mypy-cleanup

Best (and enjoy the good wine),

DJ

Op zondag 25 oktober 2026 om 18:56:25 UTC+1 schreef messi...@gmail.com:

[DJ Dekker]

Hi Johan,

I’ve opened pull requests for four open issues:

• PR #553 addresses #502. After switching from a depth/node time control to a clock-based time control, the old depth or node value remained in picochess.ini. Because these values take priority during startup, PicoChess returned to the old depth setting after a restart. Selecting a clock-based time control now clears them.

• PR #552 addresses #538. The Dutch web interface displayed English options under 'Weergave'. These dynamic menu labels are now localized. The Coach blunder alert also displays the complete Dutch word 'hint' instead of 'hnt'.

• PR #549 addresses #547. After checkmate in Analysis mode, the clock rotation returned to the previous position’s evaluation, such as mate in 1. The final result now remains visible.

• PR #551 addresses #548. PicoTutor blunder, threat, and hint messages disappeared after approximately one second. They now remain visible for their intended duration.

A suggestion for PicoChess 4.4 and the engine assets:

If you agree with the proposed changes and consider them suitable for inclusion, perhaps the next release could be PicoChess 4.4.0 rather than another 4.3.x patch release. PicoChess is currently at version 4.3.6, and the changes made over the past few weeks appear substantial enough to justify a minor-version update.

I have also prepared updated AArch64 engine packages, available for download at https://drive.google.com/drive/folders/17t_rzkG85BOZogdqPPQSfmUH75oDUSOI?usp=sharing. The Google Drive folder contains:

• engines-aarch64-lite.tar.gz
• engines-rodent3-lite.tar.gz
• aarch64_rodent4_lite.tar.gz
• README-issue468.txt
• SHA256SUMS

These packages contain Stockfish 19NN, the current NNUE version of Arasan, the updated fairy.uci, and the required Rodent III/IV files. The included README describes the changes related to #468, while SHA256SUMS can be used to verify the downloaded archives.

If the packages are accepted and published, they should resolve issues #337 and #468.

I have only updated the three aarch64 packages listed above. I have not updated any other engine packs that may exist, such as other packs attached to the 4.2.0 and 4.2.2 releases.

For a possible 4.4.0 release, perhaps the cleanest approach would be to republish all engine and resource assets under v4.4.0, including any assets that are unchanged from v4.2.0 or v4.2.2. Unchanged archives could remain byte-for-byte identical to their earlier versions.

This would make the 4.4.0 release self-contained and would allow install-engines.sh to use one central release variable:

ENGINE_RELEASE="v4.4.0"

RELEASE_URL="https://github.com/JohanSjoblom/picochess/releases/download/$ENGINE_RELEASE"

Existing installations do not automatically replace their installed engine packs when PicoChess itself is updated. Users who want the updated lite packages would therefore need to install them explicitly after updating:

cd /opt/picochess
sudo ./install-picochess.sh pico lite

The installer backs up the existing engines before installing the explicitly selected lite pack.

What do you think of this approach?

DJ

[Randy Reade]

The updated mame files are progressing through testing. This will include the chessengine plugin, mame scripts, retro.ini updates and custom artwork fixes. There's still quite a bit to do and I'm not sure how long it will take. But once complete we should have compatibility with the latest MAME releases. Claude is doing a fine job.😊

Randy

[Johan Sjöblom]

You have made a lot of progress DJ Dekker. I agree with the approach. This will clean up the issue list quite nicely. Not sure when I have time to do it, this weekend is fully booked up. Unifying all tar.gz engine files to be under the new 4.4.0 would need to be done in one sweep, updating the scripts to refer to it, and moving all files to it. Not a lot of work, it just has to be done as one step I guess.

So yes, lets go for this plan.

Files for mame etc can be replaced inside 4.4.0 whenever you Randy become ready. Depending on situation we can make it 4.4.1 which would be a cleaner approach.

BR, Johan