Remote Engine communication in Picochess

[Dirk]

Beside of the porting of the mame chess engines (I really enjoy some these old chess computers from my childhood and because  I can finally beat them :-) I am working on some further enhancements for Picochess like the PGN  game replay and „guess the next move“.

With one other specific enhanncement I need some help of the python guys here in the forum - maybe someone has an idea.

I’m  trying to change the way of the remote engine communication in Picochess so that it is a lot easier and simple to use) because I think it would be nice to play against the old chess computer and seeing the GUI of the chess computer on the remote PC (by that you can easily adjust some settings interactively etc.).

Here is an example of playing  on my Revelation 2 (not via Picochess) against my good old Chess Champion MK5  running on a windows tablet via the cool remote UCI server software  written by Ruud!

https://www.dropbox.com/s/5xznh4rpwlkwdlf/REV2_UCISERVER_MAME_ENGINES.mov?dl=0

I have implemented this in Picochess and it works perfectly fine when using a MacOS or UNIX PC but unfortunately not when using a Windows PC with Windows 10 (thanks to Al for fin ding this bug - the best tester of the world)

Here I play against my chess champion sensory challenger running on my Mac book with Picochess.

https://www.dropbox.com/s/8pcumbz8q4welbg/MESS_REMOTE.mov?dl=0

Unfortunately I can’t find the problem with windows as the remote communication happens exactly in the same way and I hope you have an idea:

First I double checked that I can SSH to the windows PC and start the engine manually.

These are the python commands for the remote communication:

First we care a remote shell object (that seems to work) and second we create the python-chess engine object based on the  generated shell object.

This command never comes back and it seems to be stuck somehow as I only get a time out error message in the end but nothing else.

(You can directly test this if you have an engine on a remote server PC) from Picochess terminal by starting python3:

#!/usr/bin/env python3

 

import spur

import paramiko

import chess

import chess.uci

################ example macOS server WORKING

>>> shell_mac = spur.SshShell(hostname='192.168.178.83', username='molli', password= ’xyz’, missing_host_key=paramiko.AutoAddPolicy())

>>> engine_mac = chess.uci.spur_spawn_engine(shell_mac, ["/Users/molli/Documents/remote_engines/remote_stkf9“])

>>> engine_mac.uci()

>>> print(engine_mac.name)

Stockfish 9 64 POPCNT

>>> 

################ windows server NOT WORKING

>>> shell_win = spur.SshShell(hostname='192.168.178.38', username= ’molli’, password= ’xyz’, missing_host_key=paramiko.AutoAddPolicy())

>>> engine_win = chess.uci.spur_spawn_engine(shell_win, ["C:\\Users\\dirkm\\Desktop\\mame_engines\\remote_stkf_win“])

 This last command never returns (until timeout) so I can’t print the engine.name  etc.

If you have any idea polls let me know…

Thanks

Dirk

[RandyR]

Dirk,

Are the double backslashes ("\\") correct? Just guessing really.

Randy

[RandyR]

Disregard.

A simple Google search answers it. (yes)

Randy

[Scally]

Hi Randy,

I tried all sorts, single & double slashes, backwards and forwards plus other settings.

I spent a couple of days on it, it’s weird as we had this working on windows with remote engines some time ago ....

I’m sure it will probably end up being something simple, we will see 😄


Cheers,

Al.

[RandyR]

Hi Al,

Can you run the file directly via a terminal on the RPi using SSH (bypassing python)? Are file extensions taken into account?

Randy

[Scally]

Hi Randy,

Yes we tried that too

Al.

[RandyR]

Hi Al,

Are you saying that you cannot run a file on the Windows PC via ssh?

Randy

[RandyR]

<sigh>

I really need to read things more closely. Dirk's original post says this works. Sorry.

I've been looking for an answer on Google, but it doesn't appear easy to do.

Randy

[Dirk]

Thanks Randy for all your feedback- appreciate it, really.

Yes, manually we can SSH to the Windows Server with same user password combination and are able to start the engine with exactly the same path/name like we have specified in the python command.

So we have really this situation:

Picochess communicates exactly via these commands at the moment using the unfortunately not up to date python-chess package withe exactly these pythons commands.

Then we have two SSH servers one one on Windows 10 and one on MacOS or Unix running.

To all Servus we can manually SSH and call the corresponding engine.

To MacOS/Unix this python communication still works but no (longer?) to Windows 10.

So no body else can establish a working SSH connection to Windows 10 with these python commands then I think it is a general issue which might have been fixed in later python-chess version for example, but that is only a guess.

Would be a pity if this remote mechanism only works with MacOS/Unix (although that is sufficient for me ;-)

Thanks

Dirk

[RandyR]

Hi Dirk.

First, thanks for all the work you are doing and have done to enhance picochess and move the project along.

A question - when you run the python3 code to start the engine on Windows, does the process show in the Windows task list? Is there anything in the Event Viewer that may indicate a problem? (just throwing out thoughts here)

Randy

[Dirk]

No nothing's shows up on the windows side - I assume the engine process on windows is not even started.

For windows the shell object can be created but when executing the creation of the remote engine object it gets stuck somewhere.

Hopefully Shiv will have time to debug the problem as well...

Do you have a PC with running SSH Server?

Then you could try the python commands by your own and see if your system behaves differently.

under python3 do the imports, create the shell and the engine object and if it does not get stuck (like in case of MACOS server) you can execute remote engine commands like engine.uci() and print(engine.name) which will eventually show the engine's name when connection is working.

Dirk

[RandyR]

Hi Dirk.

Yes, I am going to try it out. I just didn't have the time today as I was working on a different issue, unrelated to picochess. Hopefully tomorrow. 👍

Randy

[RandyR]

No luck here, either. The remote engine does not even start with spur. It doesn't even appear that spur opens a connection since other examples from the spur docs don't seem to work either. You can run a remote engine using Paramiko alone but you would have to figure out how to interact with it.

Randy

[hasnul]

Hello,

Lurker here, now "de-lurked".

This seems to be related to or is the issue cited in

https://github.com/mwilliamson/spur.py/issues/69

Setting shell type to minimal (as suggested in the issue) seems to fix it.

Example:

import spur                                                                             

from spur.ssh import ShellTypes                                                         

hostname = "h"                                                                          

username = "u"                                                                          

private_key = "pk"                                                                      

shell_type = ShellTypes.minimal                                                         

shell = spur.SshShell(hostname=hostname, username=username, private_key_file=private_key, shell_type=shell_type)

with shell:                                                                             

    result = shell.run(["hostname"])                                                    

print(result.output)

-Hasnul

[RandyR]

I tried using spur.ssh.ShellTypes.minimal but that didn’t work for me either. I even tried changing the default ssh shell on Windows 10 to Powershell but no luck. Also put the engine in the $PATH so I shouldn’t have to worry about the path syntax and still no joy. I should note that I was using Linux Mint as the ‘local’ computer.

Randy

[hasnul]

I should have mentioned earlier that the remote I was trying to connect to was running fish shell on Ubuntu.

For windows, consider tfromme's pull request (has not been merged); see

https://github.com/mwilliamson/spur.py/pull/87

[RandyR]

Yes. It works on Linux and Mac remotes. Thanks for the link. Dirk will be happy about that.

Randy

[hasnul]

Dear all,

I've managed to run x86 engines on a remote windows server without resorting to modifying the spur package.

The server is a Windows 10 Home Build 1909 (in a VM) with Windows Subsystem Linux enabled.

I disabled openssh service from the Service panel and installed ssh server for wsl.

(I don't remember opening the ssh port today; must have done it many moons ago.)

Run the ssh server in wsl: sudo service ssh --full-restart

Invoked picochess in the usual way for remote service using an ssh key.

I attempted games against the following engines:

1.Houdini 1.5

2.AnMon 5.75

3.Monarch 1.7

4.Shredder Classic 5 (commercial)

All engines are windows executable (x86 arch). 

Results were:

1. Houdini hanged as soon as it got out of book.

2. Success. Finished a game.

3. Success. Finished a game.

4. Partial success. Played two games. One finished the other one hanged in a position where Shredder could have won my Queen.

All games were played at 5:03 time control.

Regards,

Hasnul

[RandyR]

Hi Hasnul,

If I understand correctly, what you basically have is Windows running in a virtual machine on Linux, with the Windows VM itself running Linux in a VM via WSL??? Ha ha. That’s a novel solution. But, I think it would be asking too much to expect people to enable WSL just for remote engines. It’s an interesting concept, though. One thing I’m not clear about - were the engines Windows compiles or Linux compiles?

If we can get remote engines working on a native Windows computer, we could possibly run winboard/xboard engines via polyglot. 👍

Randy

[hasnul]

lol. A vm in a vm. Accidental inception ...

I didn't want to disturb the host so I ran the whole thing as a guest in virtualbox.

The 'file' program reports all the engines used as PE32 format except for Houdini -- PE32+.

They also run natively on the host of the guest vm which is also Windows 10.

- Hasnul

[RandyR]

Ok. That's great. I'm going to test the modified spur.py with the remote Win 10 with the built-in SSH Server and cmd as the shell, hopefully later today.

Randy

[Dirk]

Thank you guys for your feedback and tries.

Yes the WSL solution is way too much for the windows users.

Unfortunately the minimal shell type option does not work for me either - I think the reason is the" store_pid" option which is used by python-chess uci communication and this option is not supported by the minimal shell (and according to the readme part also not for the added modification of the windows shell I think.

Looks like we only can use Unix/MacOS for the remote engines :-(

Dirk

[Scally]

Hi all,

That’s a bummer, I thought you were onto something there.

I was looking forward to finally playing the latest free version of Komodo & some Windows only chess engines.


Cheers,

Al.

[RandyR]

I'm hopeful (over-confident?) we can get this working. 😊

Randy

[Dirk]

Me too - It would be really a pity if have to pass on windows with the remote connections.

I had several windows engines running via the WINE emulation layer on MacOS before I upgraded to th newest OS Catalina which no longer supports the 32 BIT version of WINE :-(
For example Colossus Chess by Martin Bryant (yes, there is a free UCI version for windows) or GIRAFFE, the first neural network chess algorithm(I think) implemented by one of the Google team members who developed Alpha Zero....

[RandyR]

Success!!

~$ python3

Python 3.6.9 (default, Apr 18 2020, 01:56:04) 

[GCC 8.4.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import spur

>>> import paramiko

>>> import chess

>>> import chess.uci

>>> shell_win = spur.SshShell(hostname="<ip_address>", username="<username>", password="<password>", shell_type=spur.ssh.ShellTypes.windows)

>>> engine_win = chess.uci.spur_spawn_engine(shell_win, ["D:/Arena/Engines/stockfish-11-win/sf11.exe"])

>>> engine_win.uci()

>>> print(engine_win.name)

Stockfish 220420 64 POPCNT

>>> 

I removed spur and paramiko:

sudo pip uninstall spur

sudo pip uninstall paramiko

then updated them to their latest versions (spur-0.3.21 and paramiko-2.7.1)

sudo pip install spur

sudo pip install paramiko

I then renamed /usr/local/lib/python3.6/dist-packages/spur/ssh.py as ssh.py0321

and placed the version with windows support (as suggested by Hasnul - thanks!) from the spur folder here:

https://github.com/tfromme/spur.py

I did this via the desktop so someone git savvy will have to provide the command-line equivalent. :^)

The updated paramiko solves the "CryptographyDeprecationWarning".

Note that this is from Linux Mint to Windows 10 so has not been tested on the RPi. I also don't know if the updated python modules break anything in picochess.

Cheers!

Randy

[Dirk]

Ah so cool - will have a look.

Thanks so much Randy & Hasnul!

--
You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/picochess/5092433f-0f48-4967-a644-6692253998afo%40googlegroups.com.

[Scally]

Sounds promising Guys,

Cheers,

Al.

[RandyR]

Hi all,

I tried testing an xboard engine using the Wb2Uci program (I erroneously mentioned polyglot in an earlier post) and it appears it won't work in our case. I thought you could just run the Wb2Uci.exe program from the command line (with a proper .eng file set up) but it doesn't start the 'real' engine that way. It looks like it is only designed for interfacing winboard engines to UCI GUIs, not for command line use. This was from testing directly on the Windows machine.

Oh well. Maybe someone will create a new adapter to solve this. But, the latest python-chess supports xboard engines directly.

We'll get there some day. <wink>

Randy

[Dirk]

I am pretty sure that there is a WB-> UCI adapter which is working for Picochess because I have already used one in combination with WINE on my Mac and different WB engines (but with some of some there were still some problems).
We will see..

[Dirk]

Yes it works on the PI (here a PI4) with Picochess and the warnings are gone!

Will check it with a real connection out of Picochess the next days...

Big thanks again for helping out to Randy and Hasnul!

Dirk

pi@MolliPi:~ $ python3

Python 3.7.3 (default, Dec 20 2019, 18:57:59) 

[GCC 8.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import spur

>>> import paramiko

>>> import chess

>>> import chess.uci

>>> shell = spur.SshShell(hostname='192.168.178.38', username='whoknows', password='xyzzy', missing_host_key=paramiko.AutoAddPolicy(), shell_type=spur.ssh.ShellTypes.windows)

>>> engine = chess.uci.spur_spawn_engine(shell, ["C:/Users/dirkm/Desktop/mame_engines/ccmk2"])

>>> print(engine.name)

None

>>> engine.uci()

>>> print(engine.name)

Novag Chess Champion: MK II (ver. 1) (mess 0.215)

[Scally]

Hi Guys,

Great stuff, I look forward to testing it, in the mean time I’ll update my Windows Chess Engine versions.


Cheers,

Al.

[Scally]

Hi Guys,

I’m having problems trying the same as Randy & Dirk.

I removed and reinstalled spur & paramiko

I’m on python 3.7 like Dirk, and ssh works manually and starts the engine. I’ve copied the new ssh.py file from Hasnul’s GitHub and I’m almost there, I just don’t get a response from python after starting the engine:

—————

Via SSH from RPi 4 to Win10 Laptop:

Sc...@192.168.1.200's password:
Last login: Fri Jun 12 15:22:20 2020 from 192.168.1.191
Microsoft Windows [Version 10.0.18362.836]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Program Files\OpenSSH\home\Scall>C:/Users/Scall/Documents/Picochess/Engines/stockfish9
Stockfish 9 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
uci
id name Stockfish 9 64 BMI2
id author T. Romstad, M. Costalba, J. Kiiski, G. Linscott

————-

Python3 from RPi 4 to Win10 Laptop:

Al@RPi4-SSD:~ $ python3

Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import spur
>>> import paramiko
>>> import chess
>>> import chess.uci

>>> shell_win = spur.SshShell(hostname="192.168.1.200", username= "xxxxxxxx", password="xxxxxxxx", shell_type=spur.ssh.ShellTypes.windows)
>>> engine_win = chess.uci.spur_spawn_engine(shell_win, ["C:/Users/Scall/Documents/Picochess/Engines/stockfish9"])

I don’t get the >>> to enter the last 2 commands.

I tried adding the .exe and the slashes the other way round, but nothing.

Any ideas?


Thanks,

Al.

[RandyR]

Hi Al,
Should it be 'stockfish9.exe'?

Randy

[Scally]

Hi Randy,

I tried that too, same result.

Al.

[Dirk]

Hi Al,

really strange. 

The only thing I did in addition was also to replace the local.py file besides of the ssh.py file because it was also updated - don’t believe that this will change something because it would be relevant for local connections not real remote I think.

Dirk

[RandyR]

I did that initially, too, but if I recall correctly local.py is updated in the latest spur release. When I removed spur and reinstalled, I only replaced ssh.py from tfromme's fork.

Al, does the stockfish process start on the Windows PC?

Randy

[RandyR]

Sorry about that, Al. I should have read you WHOLE post. <embarrassed>

Randy

[Scally]

Hi Randy,

Yes it starts, I also show it starting via ssh and mentioned trying separators with /, //, \, \\ and I also mentioned with and without the .exe extension.

I have a folder full of old chess engine exe’s on my laptop, all run natively and under ssh.

In case it was my folder structure I just moved komodo902 into C:\ with no extra paths, this also does not work.

I’m really stumped now. If I can’t get it working using python then I’ve no chance through Picochess.

Al.

[RandyR]

So,

engine_win = chess.uci.spur_spawn_engine(shell_win, ["C:/Users/Scall/Documents/Picochess/Engines/stockfish9"])

starts stockfish, but python3 never comes back?

I'm stumped, too.

Randy

[Scally]

Hi Randy,

Logging into the laptop from my RPi4 via ssh:

Sc...@192.168.1.200's password:
Last login: Fri Jun 12 18:48:32 2020 from 192.168.1.191

Microsoft Windows [Version 10.0.18362.836]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Program Files\OpenSSH\home\Scall>
C:\Program Files\OpenSSH\home\Scall>
C:\Program Files\OpenSSH\home\Scall>

Then typing the same path I used earlier:

C:\Program Files\OpenSSH\home\Scall>C:/Users/Scall/Documents/Picochess/Engines/stockfish9
Stockfish 9 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
uci
id name Stockfish 9 64 BMI2
id author T. Romstad, M. Costalba, J. Kiiski, G. Linscott

option name Debug Log File type string default
option name Contempt type spin default 20 min -100 max 100
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 131072
option name Clear Hash type button
option name Ponder type check default false
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 30 min 0 max 5000
option name Minimum Thinking Time type spin default 20 min 0 max 5000
option name Slow Mover type spin default 89 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name SyzygyPath type string default <empty>
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 6 min 0 max 6
uciok
quit

C:\Program Files\OpenSSH\home\Scall>

This works, however when using Python, I can’t get past the path.

engine_win = chess.uci.spur_spawn_engine(shell_win, ["C:/Users/Scall/Documents/Picochess/Engines/stockfish9"]) with and without the.exe

I shortened the path to C:/ and put an engine file in there and it still doesn’t work.

Am I missing something from Windows maybe, do I need something extra there?


Thanks,

Al.

[RandyR]

Hi Al,

Just to confirm, when you said a few posts back that stockfish9 starts, you meant that the python command is starting it, right? Not just via ssh.

Randy

[Scally]

Hi Randy,

I can only start engines via ssh

With python I get past the 4 Imports and the shell_win but get no response from the engine_win i.e. the >>> don’t return to enter the last 2 commands (engine & print)

It’s all in my first post today.


Al.

[RandyR]

I should have been clearer in my earlier post. It sounds like the ssh.py update didn't take. You've probably triple checked already. Do you have more than one version of python3 installed?

Randy

[Scally]

Hi Randy,

I have python 2.7 and python 3.7 installed only

Al@RPi4-SSD:/usr/bin $ ls py*
py3clean pybuild pydoc2.7 pygettext2.7 python2.7 python3.7-config python3m-config
py3compile pyclean pydoc3 pygettext3 python2.7-config python3.7m python-config
py3versions pycompile pydoc3.7 pygettext3.7 python2-config python3.7m-config pyversions
pyacc pydoc pygettext python python3 python3-config
pyacc-3.0.4 pydoc2 pygettext2 python2 python3.7 python3m

Al@RPi4-SSD:/usr/bin $ cd /usr/local/lib

Al@RPi4-SSD:/usr/local/lib $ ls
libleveldb.a libleveldb.so.1 libmemenv.a pkgconfig python3.7
libleveldb.so libleveldb.so.1.20 libpython3.7m.a python2.7
Al@RPi4-SSD:/usr/local/lib $

I rechecked the spur and paramiko installs a while back, and redid them.

Al@RPi4-SSD:/usr/local/lib/python3.7/dist-packages/spur $ ll
total 92
drwxr-sr-x 3 root staff 4096 Jun 12 18:24 .
drwxrwsr-x 49 root staff 4096 Jun 12 18:24 ..
-rw-r--r-- 1 root staff 1028 Jun 12 18:24 errors.py
-rw-r--r-- 1 root staff 735 Jun 12 18:24 files.py
-rw-r--r-- 1 root staff 346 Jun 12 18:24 __init__.py
-rw-r--r-- 1 root staff 2228 Jun 12 18:24 io.py
-rw-r--r-- 1 root staff 6293 Jun 12 18:24 local.py
-rw-r--r-- 1 root staff 6293 Jun 12 17:07 local.py.old
drwxr-sr-x 2 root staff 4096 Jun 12 18:17 __pycache__
-rw-r--r-- 1 root staff 1388 Jun 12 18:24 results.py
-rw-r--r-- 1 root staff 13673 Jun 12 18:28 ssh.py
-rw-r--r-- 1 root staff 12073 Jun 12 14:08 ssh.py0321
-rw-r--r-- 1 root staff 12073 Jun 12 14:07 ssh.py.old
-rw-r--r-- 1 root staff 200 Jun 12 18:24 tempdir.py
Al@RPi4-SSD:/usr/local/lib/python3.7/dist-packages/spur $

Much earlier without the ssh.py from Hasnul I couldn’t get past the shell_win command, once I got past that I thought I was there, but sadly not.

Al.

[RandyR]

I have python 3.7.3 on my RPi4. I just tried it and it's working for me. It sounds like there is something amiss with your OS. Do you have a different SD card to try?

Note that I am not running a virtualenv.

Randy

[Scally]

Hi Randy,

I have my main Picochess system running on a RPi 4, I’ll try there.

This was on my SSD boot RPi 4 which shouldn’t really make a difference.

I’ll let you know.

Thanks,

Al.

[RandyR]

Hi Al,

I agree. Not sure what's going on there. I know you'll solve it!

Randy

[Scally]

Hi Randy,

Ok I’ve installed a brand new RaspberryPi OS Lite image.

It has python 2.7.16 & python 3.7.3 installed

Everywhere I read it says I need to install pip3 for python3 and pip for python2, yet your instructions say ‘pip install spur’ not ‘pip3 install spur’ which python3 would need.

Picochess uses pip3, so what do I need?

I’ve tried both:

pip3 installs spur but the folder is not in /usr/local/lib/python3.7/dist-packages so I can’t add the modified ssh.py
pip produces numerous error messages on trying to install spur, with or without it being installed with pip3

Have you done an install from a new image, if so, what do I install and in what order?


Thanks,

Al.

[Scally]

.... Just to be clear

I have not installed Picochess or anything else, I’ve started with a fresh image.


Al.

[Dirk]

Hi Al,

my Picochess installation is also based on a fresh Buster light image and I used pip3 to remove and install spur/parmiko.

I could find the folders and files und python3.7/dist-packages.

Dirk

pi@MolliPi:/usr/local/lib/python3.7/dist-packages $ ls -all
total 404
drwxrwsr-x 54 root staff 4096 Jun 11 18:35 .
drwxrwsr-x 3 root staff 4096 Aug 12 2019 ..
drwxr-sr-x 3 root staff 4096 Jun 11 18:35 altgraph
drwxr-sr-x 2 root staff 4096 Jun 11 18:35 altgraph-0.17.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 bcrypt
drwxr-sr-x 2 root staff 4096 Aug 12 2019 bcrypt-3.1.7.dist-info
drwxr-sr-x 3 root staff 4096 Jun 11 14:13 berserk
drwxr-sr-x 2 root staff 4096 Jun 11 14:13 berserk-0.10.0.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 certifi
drwxr-sr-x 2 root staff 4096 Aug 12 2019 certifi-2019.6.16.dist-info
drwxr-sr-x 4 root staff 4096 Aug 12 2019 chardet
drwxr-sr-x 2 root staff 4096 Aug 12 2019 chardet-3.0.4.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 chess
drwxr-sr-x 3 root staff 4096 Aug 12 2019 click
drwxr-sr-x 2 root staff 4096 Aug 12 2019 Click-7.0.dist-info
drwxr-sr-x 2 root staff 4096 Aug 12 2019 ConfigArgParse-0.14.0.dist-info
-rw-r--r-- 1 root staff 41174 Aug 12 2019 configargparse.py
drwxr-sr-x 2 root staff 4096 Aug 12 2019 configobj-5.0.6.dist-info
-rw-r--r-- 1 root staff 89627 Aug 12 2019 configobj.py
drwxr-sr-x 4 root staff 4096 Apr 8 21:10 Deprecated-1.2.8-py3.7.egg
-rw-r--r-- 1 root staff 92 Jun 11 14:12 easy-install.pth
drwxr-sr-x 4 root staff 4096 Aug 12 2019 flask
drwxr-sr-x 2 root staff 4096 Aug 12 2019 Flask-1.0.2.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 idna
drwxr-sr-x 2 root staff 4096 Aug 12 2019 idna-2.8.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 itsdangerous
drwxr-sr-x 2 root staff 4096 Aug 12 2019 itsdangerous-1.1.0.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 jinja2
drwxr-sr-x 2 root staff 4096 Aug 12 2019 Jinja2-2.10.1.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 markupsafe
drwxr-sr-x 2 root staff 4096 Aug 12 2019 MarkupSafe-1.1.1.dist-info
drwxr-sr-x 5 root staff 4096 Aug 12 2019 nacl
drwxr-sr-x 4 root staff 4096 Apr 8 21:10 ndjson-0.3.1-py3.7.egg
drwxr-sr-x 3 root staff 4096 Aug 12 2019 OpenSSL
drwxr-sr-x 3 root staff 4096 Jun 11 08:20 paramiko
drwxr-sr-x 2 root staff 4096 Jun 11 08:20 paramiko-2.7.1.dist-info
drwxr-sr-x 6 root staff 4096 Aug 12 2019 pyasn1
drwxr-sr-x 2 root staff 4096 Aug 12 2019 pyasn1-0.4.6.dist-info
drwxr-sr-x 2 root staff 4096 Aug 12 2019 __pycache__
drwxr-sr-x 12 root staff 4096 Jun 11 18:35 PyInstaller
drwxr-sr-x 2 root staff 4096 Jun 11 18:35 PyInstaller-3.6.dist-info
drwxr-sr-x 2 root staff 4096 Aug 12 2019 PyNaCl-1.3.0.dist-info
drwxr-sr-x 2 root staff 4096 Aug 12 2019 pyOpenSSL-19.0.0.dist-info
drwxr-sr-x 2 root staff 4096 Aug 12 2019 pyserial-3.4.dist-info
drwxr-sr-x 2 root staff 4096 Aug 12 2019 python_chess-0.22.1.dist-info
drwxr-sr-x 3 root staff 4096 Aug 12 2019 requests
drwxr-sr-x 2 root staff 4096 Aug 12 2019 requests-2.21.0.dist-info
drwxr-sr-x 6 root staff 4096 Aug 12 2019 serial
drwxr-sr-x 3 root staff 4096 Jun 11 08:20 spur
drwxr-sr-x 2 root staff 4096 Jun 11 08:20 spur-0.3.21.dist-info

[RandyR]

Hi Al,

I used pip when I was testing from my desktop linux (Mint 19.3) install. I believe it soft-links to pip3 and installs modules under python3 (/usr/local/lib). To force an install under python2 you would use pip2.

On the RPi4, the Buster image was 2020-05-27-raspios-buster-lite-armhf.img and picochess was installed on it. In this case I did use pip3 to remove and reinstall spur and paramiko, then replaced ssh.py in /usr/local/lib/python3.7/dist-packages/spur, just like Dirk shows.

I'm going by memory and will double-check the RPi4 bash history to be certain and reply if I was in error.

The Windows machine is using Build 1909 of Windows 10 Pro (I see yours was 1903) but I doubt that would make a difference. However it could be something else on the Windows PC that's giving you grief. Also, it's the version of OpenSSH that comes with Windows 10, if that matters.

Randy

[RandyR]

Interesting reading...

https://snarky.ca/why-you-should-use-python-m-pip/

Randy

[Scally]

Hi Dirk & Randy,

Thanks for you continued help on this .......

OK I managed to install everything required on a new RPi image just to test this bit, and again it freezes with no reply to engine_win command

So it must be my laptop that is at fault.

I’m not a Windows expert, but I do remember activating the internal OpenSSH when I tested the original remote engines (which worked)

I’m not buying a new laptop just for this, so if there’s anything you guys can think of that may help on Windows 10 please let me know.


Thanks,

Al.

[RandyR]

Hi Al,

I'm not sure what antivirus or firewall you have installed on Win 10. I'm using BitDefender so Windows Defender is disabled. You could try disabling your AV/Firewall. Not sure this would be the issue since you can use ssh manually and it works.

A refresh of Windows 10 could be worth trying, or if you have things backed up, a clean install of Windows 10. Maybe, before doing that, just do the windows updates and keep doing them until you are offered the next build (1909 or 2004) and install that.

Randy

[Scally]

Hi Randy,

I should have said, it’s fully updated and I turned off AVG to allow the ssh etc.

So it’s nothing obvious. I bought this laptop for chess only so it hasn’t got much on it apart for several images and chess engines.

Al.

[RandyR]

Hi Al,

Thanks.

From an earlier post, your manual ssh into Windows showed this:

C:\Program Files\OpenSSH\home\Scall>

That doesn't look like the default OpenSSH that comes with Windows. Did you install OpenSSH separately? If so, perhaps uninstall it and enable the default one via Settings/Apps/Optional Features/Add a Feature/OpenSSH Server. Reboot. Then go to services and start the OpenSSH Server service.

My prompt looks like this:

randy@RR-TABLET C:\Users\rrvyp>

Randy

[Scally]

Hi Randy,

Eureka !!

Yes, I had OpenSSH installed, so I removed it.
Unfortunately OpenSSH Server was not an option in the ‘Add a feature’ however our friend ‘google’ finally showed me the way using Power Shell etc to activate the inbuilt OpenSSH.
With a few tweaks here and there, this happened:

Al@RPi4-SSD:~ $ python3
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import spur
>>> import paramiko
>>> import chess
>>> import chess.uci
>>> shell_win = spur.SshShell(hostname="192.168.1.200", username="xxxxxxxx", password="xxxxxxxx", shell_type=spur.ssh.ShellTypes.windows)

>>> engine_win = chess.uci.spur_spawn_engine(shell_win, ["C:/Users/Scall/Documents/Picochess/Engines/stockfish9.exe"])
>>> engine_win.uci()
>>> print(engine_win.name)
Stockfish 9 64 BMI2
>>>

So great spot with OpenSSH being the wrong version and thanks very much for your help. 👍🖖


Cheers,

Al.

[RandyR]

That’s great, Al. Glad you didn’t have to reinstall Windows. 👍

Randy

[Dirk]

Hello,

remote connection to windows also works from picochess itself (at least with my current dev version ;-) - great, thanks again for helping.

There is one difference to my Mac connection which I would like to understand/change if possible.

One use case for the remote engines (despite the fact that there are some windows/windboard board engines which we don't have on our Pi i) is to run mame emulated chess machines so that we have the GUI of the machine and are able to change some settings which are not possible via the UCI settings in picochess.

On my Mac when I start the mame remote engine the engine window pops up and I can change and input all possible settings there.

But with windows the mame task is running completely in background with no windows popup at all, see my already posted video:

https://www.dropbox.com/s/8pcumbz8q4welbg/MESS_REMOTE.mov?dl=0

Does anybody know how to start the remote shell processs (I start a bash script to call the mame engine) allowing it to open windows (in the child process)?

That would be great - otherwise calling the mame engines remotely wouldn't make sense as we can now runtime directly on the Pi.

This is how my windows bash file looks like which is started via ssh:

cd C:\Users\dirkm\Documents\remote_engines\messchess

C:\Users\dirkm\Documents\remote_engines\messchess\MessChess.exe ccmk2

Any ideas?

Dirk

Am Donnerstag, 11. Juni 2020 10:39:00 UTC+2 schrieb Dirk:

Yes it works on the PI (here a PI4) with Picochess and the warnings are gone!

Will check it with a real connection out of Picochess the next days...

Big thanks again for helping out to Randy and Hasnul!

Dirk

pi@MolliPi:~ $ python3

Python 3.7.3 (default, Dec 20 2019, 18:57:59) 

[GCC 8.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import spur

>>> import paramiko

>>> import chess

>>> import chess.uci

>>> shell = spur.SshShell(hostname='192.168.178.38', username='whoknows', password='xyzzy', missing_host_key=paramiko.AutoAddPolicy(), shell_type=spur.ssh.ShellTypes.windows)

>>> engine = chess.uci.spur_spawn_engine(shell, ["C:/Users/dirkm/Desktop/mame_engines/ccmk2"])

>>> print(engine.name)

None

>>> engine.uci()

>>> print(engine.name)

Novag Chess Champion: MK II (ver. 1) (mess 0.215)

>>> 

Am Dienstag, 9. Juni 2020 21:48:06 UTC+2 schrieb RandyR:

Success!!

~$ python3

Python 3.6.9 (default, Apr 18 2020, 01:56:04) 

[GCC 8.4.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import spur

>>> import paramiko

>>> import chess

>>> import chess.uci

>>> shell_win = spur.SshShell(hostname="<ip_address>", username="<username>", password="<password>", shell_type=spur.ssh.ShellTypes.windows)

>>> engine_win = chess.uci.spur_spawn_engine(shell_win, ["D:/Arena/Engines/stockfish-11-win/sf11.exe"])

>>> engine_win.uci()

>>> print(engine_win.name)

Stockfish 220420 64 POPCNT

>>> 

I removed spur and paramiko:

sudo pip uninstall spur

sudo pip uninstall paramiko

then updated them to their latest versions (spur-0.3.21 and paramiko-2.7.1)

sudo pip install spur

sudo pip install paramiko

I then renamed /usr/local/lib/python3.6/dist-packages/spur/ssh.py as ssh.py0321

and placed the version with windows support (as suggested by Hasnul - thanks!) from the spur folder here:

https://github.com/tfromme/spur.py

I did this via the desktop so someone git savvy will have to provide the command-line equivalent. :^)

The updated paramiko solves the "CryptographyDeprecationWarning".

Note that this is from Linux Mint to Windows 10 so has not been tested on the RPi. I also don't know if the updated python modules break anything in picochess.

Cheers!

Randy

[RandyR]

Hi Dirk,

From what I've been reading it's not easy to do on Windows. You probably read the same thing. It looks like Microsoft doesn't want windows opening up all of a sudden on logged-in users' screens. It looks like Windows treats logins as different sessions even when they are using the same username/password. Microsoft provides tools to do it, but these run on a Windows PC, not linux (I'm referring to PSExec).

There is a linux program called 'winexe' that might do what you want. I don't know if it would work on a remote Windows 10 OS, though.

Randy

[Dirk]

Yes Randy - I think you are right: windows does not want to allow popup windows in remote session (which absolutely makes sense).

I thought there might be an option to allow this explicitly but it looks there isn't any.

Thanks for having a look, Randy!

Dirk