Where did JRomang get the information about the DGT-board and clock communication?

[mvanthoor]

Hi,

After a long while I'm looking into returning to chess programming again. (Let's just say it was a hectic 18 months at work with two massive projects.)

First: Does anybody know where J. Romang got the information with regard to the board and clock communication?

The reason why I'm asking is that I'm looking into writing a replacement for PicoChess in Rust, because:

1. I detest Python. I just don't want to work with that language for several reasons. (It being interpreted being one of them; the messy syntax is another reason, and constantly breaking stuff on new versions is a third; and that is just the tip of the iceberg.)

2. Even though I've got great respect for what has been accomplished with regard to PicoChess, let's just say I'm not really a fan of the coding style.

So therefore I'm looking into extracting the communication protocol somehow and then rewriting a fully compiled, multi-threaded async replacement version in Rust.

It will be open-source, people can either use it or not, contribute or not; I don't mind. I'm going to write this for me, myself and I, just like I did with my own chess engine, and will do with new Fritz11-like cross-platform user interface.

Second: I now have a second DGT-board. I've installed Picochess (0.9N, the last "official" version) on a 64-but NUC under Debian 11. (So, not Raspberry Pi.) It works, but it doesn't boot reliably; neither with systemd, nor directly from the command-line. Sometimes it gets stuck. It says "USBboard", and then doesn't continue; it either never continues, or only after a very long while. On the command-line, it says: "Warning: clock has been locked for more than 2secs." If it doesn't get stuck, it 'just works'; I have not been able to find what causes this problem.

This is one of the newer wooden USB-C boards. My older Mini-USB board does not seem to have this problem; the same system always reliably starts. Does anybody have more information about this issue?

The DGT Pi runs Picochess 0.9L, which is even older than the 0.9N I'm using, so I can't imagine that the code has to be changed to work correctly with the USB-C boards.

The difference between the boards is that one sits on /dev/ttyUSB0 (the Mini-USB one), and the other sits on /dev/ttyACM0 (the USB-C one). So they're both serial ports, but they're not of the same class. If programmed correctly, this should not make a difference, but obviously, somehow, it does.

It doesn't matter if I start PicoChess with the -d <port> option or without it.

Any insight would be appreciated, should someone else have encountered this.

[mvanthoor]

Th image I've created myself in 2020, running on Debian / Raspbian Buster with PicoChess 0.9n, has the same problem on the USB-c board. It works reliably on the Mini-USB board, but on the newer USB-c board Picochess sometimes either doesn't start, or it just doesn't seem to get through to the clock. (No beep, no message.) Sometimes, it sits at "USBboard"  for about 1.5 minutes.

I also have a second DGT 3000 Limited Edition with this second board; I'll test that too; just as the older DGT XL I also still have.

This is frustrated. I've had this board for quite some time now (over a year) but no time to really use it; now that I start using it, it seems to not work correctly with PicoChess, and I have no information on the communication protocol to check if everything going in and out of the board is correct.

[mvanthoor]

I found that, if Picochess beeps and then gets stuck at "USBboard", it will continue after I switch the clock off and then on again.

Sometimes however, Picochess doesn't seem to start (no beeps, no messages), but that isn't true; when I log in through ssh, I can see Pichochess running.

Found this thread, with similar problems: https://groups.google.com/g/picochess/c/020YSNjipqo

PicoChess 0.9L in the DGTPi has to work with all of the boards, so tomorrow I'm going to download that image, flash it, disable the DGTPi setting and remove the DGTPi-specific services, and see if it works. If so, I can compare that image's setup and/or init code to the one in my 0.9N version.

[mvanthoor]

Last attempt to gain more information also failed.

I disabled Picochess, then restarted the computer, and started the systemd service manually. Nothing seemed to happen. No beeps, no message on the clock. I restarted the service with 'sudo systemctl restart picochess.service'; then I got the beep, but it hung for 2 minutes at 'USBboard' before continuing.

This way, PicoChess is unusable with a newer e-board (and probably SmartBoard); it could work with the DGTPi, because I just remembered that it is wired differently; through the Pi's I2c pins instead of through the board. So I don't even have to check the code in the DGT Pi image because it'll probably be the same as the 0.9N code. (The DGTPi needs the additional dgtpicom service if I remember correctly.)

In the other thread I saw someone having success with a different cable (?) I tried three USB cables and three audio cables, including the originals, without any difference.

[mvanthoor]

Another finding...

If the board+clock combination gets to work (i.e. start the computer, turn off the clock and then on again, and if need be restart the computer), then picochess and even the computer can be restarted as many times as I want; it will always work. Start->Beep->display USBboard -> Time -> Play.

Tomorrow I'm going to e-mail DGT to see if this can be resolved somehow, either in the PicoChess code through a different series of initializations, or a new firmware; if even that is possible with the USB non-BT board. (I'm also going to ask them if there are communication specs for the incoming/outgoing data on the serial port. Someone must have had them at some time... or reverse engineered them somehow.)

[RandyR]

Hi Marcel,

This might be of some use for you. Also, this thread has some further information. It's definitely a USB connection issue. I assume your clock is connected to the board using the clock cable. Nothing will get to the clock until the USB connection is made. When it's not working, is there a /dev/ttyACM0 device created?

Randy

[mvanthoor]

Hi Randy,

Thanks, I'll look through those threads. I'll also send a mail to DGT to see if I can get the serial specifications of all DGT boards. That would make implementing a new library much easier than distilling everything from PicoChess and the old DGTNix/DGTdrv projects. (And I'm still curious where THEY got the specifications.)

It doesn't seem to be a connection issue with the USB-port. /dev/ttyACM0 is always created if the board is connected. I've also tried delaying the startup of PicoChess (thinking it may be initializing before ACM0 is created), but even a delay of 30 seconds doesn't help. What also leads me to believe that it is not a USB issue is that PicoChess starts, then the clock beeps and shows "Pico 0.9n" and "USBboard" (so the USB-connection is fine) and then it hangs with the message "Clock is locked for more than 2secs". I think that the newer boards, after Picochss sends "USBboard", require some action/command to get the clock unlocked. (If this is the board's responsibility and it doesn't do it, then it could be considered a bug in the firmware.) When I power cycle the clock (off->on), PicoChess continues and then works normally.

So I definitely think it is not a USB connection issue, but PicoChess running afoul of a different way of handling the clock by the newer boards.

[Randy Reade]

I agree. If /dev/ttyACM0 is being created, the Pi is connecting to the board, but PicoChess is unable to communicate with it, hence the USB-Board prompt on the display. It's too bad Lucas doesn't work for DGT any more. He did mention that the PicoChess code would have to change to support the new boards. As for the locked clock, I see that when the clock isn't connected but not really sure what it means - I can't decipher the python code.

Maybe you can try getting in touch with Jürgen Precour.

Randy

--
You received this message because you are subscribed to a topic in the Google Groups "PicoChess" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/picochess/htsbM79amNA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to picochess+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/picochess/c639328e-8a21-4de6-9642-cd54b1baaeb1n%40googlegroups.com.

[mvanthoor]

Hi,

OMG... it seems that the protocol is described in dgtbrd_ruud.h. How could I have missed that file, after trouncing through PicoChess for hours yesterday?! Too tired, probably...

PicoChess is able to communicate with the board, but right after "USBboard" it hangs until the clock is power cycled. Therefore PicoChess seems to be waiting for the board to do something or send it a message. Maybe it is enough to just print USBboard, wait a second, and just continue.

I can decipher the Python code if I wanted to, but probably I'm not going to. To be really honest, PicoChess is a mess, code-wise. Some files are 1200 lines long, and it-then-else statements indent 6-7 levels deep. On top of that, Python is an (almost) untyped dynamic language, so you can basically do whatever you want. I don't want to touch it.

As I said: I've got great respect for what has been achieved with PicoChess, but I don't like the PicoChess code or the fact that it is written in Python. This is the plan:

- I have my own chess engine written in Rust, called Rustic. I'm going to extract the board representation, move generator, etc... into a library; possibly called rustic_lib. The engine itself will at some point use this library. This would be the replacement for the chess / py_chess library in python. (I could use Schakmaty, which is already such a library written in Rust... but I don't see a reason to because I have my own code already.)

- On the basis of that protocol file you linked, I'll try to write a library that implements the low-level DGT protocol. This would be the open source brother of DGTBRDDLL.dll (maybe even use the same function names so it could be a drop-in replacement). Rustic supports both UCI and Xboard protocols; so it can load either one or the other when it starts. This library can be written in the same way, so it could be extended with multiple protocols for different chessboards.

- After that, a command-line application / daemon should be written that combines both of the above libraries, and it would need to implement the UCI-interface to load engines. This, effectively, would be a Rust-based replacement for PicoChess.

All of this will probably take a few... years... to write. Unfortunately I'm not retired yet. That will also take a few ... decades (but with current rules in the Netherlands, this could be easily extended to centuries).

For now I'll use my older DGT-board for playing in the dining room, against PicoChess 0.9N on the NUC, and use the newer USB-c version to develop this new stuff. It seems I at least know where to start now, but I'll send a mail to DGT for some more information. It would be great if they could at least provide information on what changes need to be made for the new boards.

I don't have the BT board; I don't know if I ever will, because I find BT a finicky type of connection. (I've read in some threads that revision 1.4 of the RPI4 can't connect, where revision 1.2 can.) First, I'll start on the library to get both the mini-USB and USB-c board and the DGT3000 and XL clock going. The older USB-Hirose board also had an FTDI chip, so I assume it is exactly the same as the mini-USB board. I had that board, but I sold it to upgrade to the USB-c board. As said, I don't know if I'll ever own the BT-board, and I definitely won't own the Revelation II unless DGT gives me one for free. Beatiful chess computer, but at €2750, it's about €1250-€1500 more than I'm willing to spend on such a device.

The DGTPi.... well... I like the concept, but I don't like the fact that it needs auxiliary C-code to be able to make the clock connection work, and it's very finicky with regard to the speed it has to run at. As this is a complete DGT-product with a manual, I doubt if anyone cares which software it runs, besides the engines, as long as it works as is described. So I don't think I'll be getting a DGT Pi either.

Kind regards,

Marcel

[RandyR]

Marcel, that's awesome that you are taking on this challenge. PicoChess in its current form is great for what it promises - play chess against engines using a DGT board. Everyone who had a hand in developing and then enhancing it over the last few years deserve much credit. Sure, the code is messy, but it's freely available and runs on low cost hardware in a free OS. I don't know anything about Rust, and barely follow Python code, but it sounds like you have the skill to pull this off.

Good luck.

Randy

[mvanthoor]

Hi Randy,

I just hope PicoChess can somehow be fixed to work correctly with the newer boards. The file you linked to runs from 2009 to 2017; I hope DGT can/will provide a newer file. I'll e-mail them tomorrow. (I am assuming that this file is not in PicoChess by accident and that somehow, somewhere, DGT either gave it to someone or had it on the internet. I did find an old 2016 developers guide through the way back machine, which stated several times things as: "If you want to .... then contact support.") Aart Bik also has the Chess for Android app, which supports DGT, Millennium and Certabo, so I suppose he obtained protocols directly from the manufacturer. Reverse-engineering all that would take huge amounts of time.

I'd be willing to at least look into fixing PicoChess so it works with my newer board.

I think I might call the library "ECBI": Electronic Chess Board Interface. It would incorporate a protocol in the same way as Rustic does for UCI and XBoard, so this library could potentially support multiple boards in the future.

Certainly I have the skills to pull this off; I hope I also have the time (and energy) to actually do this after a day's work programing on a huge business application. The last 12-18 months were absolute pandemonium in readying this application for a huge rule change by the government starting at Jan 1 2022. That is done now, but we're probably going to replace this 15 year old application by something new now.

[RandyR]

On Sunday, January 23, 2022 at 12:20:44 PM UTC-6 RandyR wrote:

I agree. If /dev/ttyACM0 is being created, the Pi is connecting to the board, but PicoChess is unable to communicate with it, hence the USB-Board prompt on the display. 

My mistake. I was thinking of the 'no e-board \' prompt. So, it looks like it does at least see the USB-C board. Hopefully DGT can help.

Randy 

[mvanthoor]

Hi Randy,

I've e-mailed them today with the question about the USB-C board+clock combination; if it needs something different to be done compared to the mini-USB+clock combo. (PS: I could test the USB-C board with the older XL clock... I have that lying around from the past as well. I didn't sell that when I sold my Hirose board.) I've also requested if they can send me the latest version of the low level serial communication. The one you linked stops in 2017; maybe there's a newer one, that also incorporates the commands that were listed by Lucas in your other linked thread from above. It would be nice to have a complete and up-to-date spec file. It would make fixing Picochess possible (I also stated that in the e-mail), and it would make developing a new program easier.

Kind regards,

Marcel

[mvanthoor]

Nice...

DGT reacted to my support question. They sent me a massive 11 MB pdf-file (I assume it's because of the logo's and pictures...) that contains a complete description of all the capabilities of the e-board and clock protocol. They also stated that "there is no difference between the protocol for the newer SmartBoards and USB-C boards."

If so, this would mean one or both of the following:

1. The older Hirose and Mini-USB  e-board connected through /dev/ttyUSB0 using an FTDI serial-to-USB chip. The newer USB-C board connects trhough /dev/ttyACM0, using a Hitachi chip. This may make some sort of difference which PicoChess does not take into account, or;

2. PicoChess does something wrong with regard to the protocol and it happens to work on the older boards, but causes a problem on the newer boards.

I may look into this and see if I can debug it, but to be honest, I don't relish debugging Python-code. It reminds me too much of PHP... with the addition of having no braces and a horrible syntax. But I'll see if I can somehow fix at least the USB-C board.

Also I'll start looking into writing a "Hello World" example that runs on both the older Mini-USB board, the newer USB-C board, and with the DGT3000 and XL clock. Hello world as in: make the clock beep, display something, and beep again. If I get it that far, I'm sure I can get the rest done as well, given enough time. After I have this done and create the repository on GitHub, I'll post it in this group. People who are interested can follow developments there.

Note: this will not be a PicoChess port from Python to Rust; it's going to be a new project. It will, however, be heavily inspired by PicoChess. The main perk will be that Rust is a MUCH faster language and it encourages (IMHO) better coding practices than Python.

With regard to both fixing PicoChess for the USB-C boards and starting the new project: don't hold your breath. It depends on the amount of free time I'll have to actually work on stuff such as this. (And I also have my own chess engine Rustic to develop, obviously, and other hobbies.)

I don't have a BT board, so I'll be completely unable to test or fix anything for those. Same for the DGT Pi (should it have any issues.)

[Randy Reade]

Marcel,

Have you tried just using the web interface in a browser, with the clock unplugged?

Randy

--
You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/picochess/744dbea5-7f3b-4eb3-8b57-bbcc601f81fdn%40googlegroups.com.

[mvanthoor]

Hi Randy,

No. I don't want to use it for move and time disaply because I find the web interface too busy. I could suppose and test it without the clock though, to see if PicoChess runs without the clock. I could also test it with DGT XL. However, in the end, it has to work with the DGT 3000.

My own system would also have a webinterface, and then use this as the main way of displaying times and moves; either through a screen attached to the computer, or through a different device accessing the web interface. Same as PicoChess does, but with a less busy GUI.

Kind regards,

Marcel

[mvanthoor]

By the way: does version 3.0 have fixes for newer boards or clocks already, or are they exactly the as 0.9N with regard to communication, with exception of the bluetooth patch ? Where could I find that BT patch by the way? There has also been a patch for dgtpicom some time ago; AFAIK, there's no documentation on which patch was incorporated in which version, or is there?

[Randy Reade]

I'm not sure if anything was changed as far as the board communications is concerned. Dirk Mollmann would have to answer that. I believe it's all in the board.py file in the picochess/dgt folder. I think the BT patch you are referring to is the one in the picochess/scripts folder for the RPi4b. But I don't think it solves BT for everyone. There's a separate thread where I found a fix (well, workaround might be more accurate) for BT connection issues using current versions of BlueZ. It works very well but takes a bit to set up. Finally, dgtpicom is only used for the DGTPi or a modded DGT3000. It's included in my v3 images and on Dirk's Github site.

Randy

[Claude Viéville]

Hi everybody,

I'm new in this conversation. I'm looking for informations on how to connect only a old dgt xl clock to a computer (in fact to a esp32) without eboard. 

Are the two wires of the jack cable serial asynchronous channels one for input the second for output ?

If yes, what is the speed (bauds rate - number of bits - wit or without parity) ?

I would really appreciate an answer on these topics.

Thank you very much.

Have a nice day,

Claude