Adding Horsepower

[J P Watters]

Alan and Steve,

In the last two months the current solar environment has seemingly lessened the number of spots that I have been seeing. Two months ago, it would have been 25K spots a day. To add to the saga, I had been running SparkSDR on a aged Apple Mac Mini. So today I assembled a cluster of 5 Raspberry 4 8GB Pi computers from the geeks at PicoCluster. I intend to connect the pair of HL2b9 SDR for skimming. 

Not knowing about running any kind cluster software, PicoCluster has it's own version. I am going to speculate that installing SparkSDR 2.0.3.5 would be a way to skimming with more horsepower. Or maybe there are other softwares that would be better choices.

So if there is any guidance to skim FT8, or to skim a little and leave a receiver or two for tinkering with a QSO or two at a time.

Looking forward to loading software and seeing what a cluster  will do for performance. 

..jpw J P Watters

KC9KKO

Morris, IL USA

[Steve Haynal]

Hi Alan and JP,

I don't know if SparkSDR is setup for cluster processing, but maybe we can talk Alan into adding some support for ZeroMQ to SparkSDR. This is what I am using for my homegrown ft8 decoder. We would just need to agree on a message format for 15 seconds of 4kHz IQ data and then another response message for the decodes. There are C/C++/C# bindings and the library is pretty straight forward to use. I can provide Python examples of what I am doing if helpful.

73,

Steve

kf7o

[Alan Hopper]

Hi Steve & JP,

there is food for thought there. I've just integrated FT4 and FT8 decoders into spark based on the ft8_lib code.  They are very much faster than  wsjtx but not as sensitive. I'm interested in trying other decoders.  In integrating the ft8_lib code I separated  the message unpacking from the decoding i.e. I pass up the 77bit code rather than the jt9.exe style decoded message string, this saves re parsing the message to detect callsigns etc.

The python decoder from the links you recently posted looks more sophisticated than ft8_lib, do you have any measure of its speed and sensitivity?

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

I haven't done head-to-head comparisons, but my feel is that the speed of Pavel's decoder is much faster, and the number of decodes is the same provided it is set to depth 3.

Currently I use zmq to send a job which consists of a complex numpy array containing the 32-bit floats of samples for the 15 second window, plus a unix timestamp of the end of the cycle and a integer dial frequency in Hz. I pickle Python objects, but we could agree on standard C arrays and orders for these. I can also return 77bit spot information, perhaps as a character arrays. You can see what I am doing here:

https://github.com/softerhardware/Hermes-Lite2/blob/master/software/ft8/hl2zmq.py

https://github.com/softerhardware/Hermes-Lite2/blob/master/software/ft8/ft8.py

It takes only a dozen or so lines of code for the zmq send or receive. I am using a ROUTER pattern to push data, and a PULL pattern to receive spots. I think there are C# (I assume that is what you are using?) examples on the zeromq site.

I think you also added web sockets to SparkSDR recently. Could this be used for external decoders?

73,

Steve

kf7o

[Alan Hopper]

Hi Steve,

I'll have another look at Pavel's decoder, I thought it was basically the wsjtx one with iq input, does it have further optimizations?

Another advantage of passing the 77bit packet is that it allows control over hashed callsigns which I have ignored up until now.

The websocket connection could be used for decoders, it already has an iq feed option and I mean to add a sample rate setting for it, zmq and gRPC are other options that I can easily support, I'm sure others will be interested in connecting to decode servers so they may have views.

The client server mode that is almost ready will allow running many copes of spark on different computers and share the bandwidth of whatever radios are connected, this effectively allows 'cluster decoding'.

Too many options, not enough time :)

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

I reviewed Pavel's ft8d changes in github as his code is based off of wsjtx from about a year ago and I wanted to see how easy it would be to use the latest wsjtx. There are actually quite a few changes and optimizations, such as switching to 4kHZ IQ data and using pfft. Your client server mode sounds interesting and may be just the thing for cluster decoding.

73,

Steve

kf7o