SparkSDR meets DXCluster

[AndyZ K1RA]

Do you digital-mode (FT8, FT4, JT9...) DX? Do you utilize any of the public DXCluster telnet servers to watch for new stations from different countries, states, grids, etc. across different bands? Do you hook into those servers to aggregate and alert on DX spots with apps like RXClus? 

Those world-wide telnet servers are great for checking band conditions, but not always useful or a good indication that you are actually receiving said spots at your local QTH.  What if you could alert only on stations decoded on any band+mode that were actually received at your QTH using your multi-band, multi-mode SparkSDR server? 

For the moment, if you are a Windows user, now you can create your own local, DXCluster like telnet server that will feed any telnet client all the spots received and decoded by your SparkSDR using my SparkSDR2DXC app.

A few notes... I've only been testing this with v2.0.3.9 Win64, with General Settings > Station > Web Sockets = Enabled. but this may work with newer versions. You must load and run the app on the PC where your telnet client resides, though it does not need to run on the same PC as your SparkSDR server.  I've tested running my app and telnet client on my PC, while hosting SparkSDR on my laptop.

Get the sparksdr2dxc.exe app from my Google Drive here:

https://drive.google.com/file/d/1xhoOb7ISHz1IZG9hemlVf7O67FKalVMK/view?usp=sharing 

It's stand alone and can be saved/copied to any folder.  To test it, you'll need to open up a CMD command prompt, navigate to the folder/directory where you saved the app and start it as follows:

sparksdr2dxc CALLSIGN SPARKIP

where CALLSIGN = your call and SPARKIP = the IP address (aaa.bbb.ccc.ddd) of your SparkSDR server. If everything is on one PC, then you can use 127.0.0.1

If the app is working you will start seeing decodes show up in that CMD window. This will show EVERY decoded station on every band sent by SparkSDR.  To exit the app just use Ctrl-C.

The next step is to test with a Telnet client while the app is stall running. You can do this with any 3rd party app like PuTTY, or the built-in Windows Telnet client, which may require going to Control Panel > Programs & Features > Turn On Windows Features and check Telnet Client.  OR of course you can go for broke and just point RXClus at the SparkSDR2DXC app, which should be host = localhost (or 127.0.0.1) and TCP port = 7373.

For testing with the Windows telnet client open up another CMD prompt and type the following;

telnet 127.0.0.1 7373

These decodes aren't as frequent as the main app window you saw earlier.  I cache unique call+band combos for 5 minutes and will not send again if I see them in that period of time.  The call+band cache expires and if the call is seen again after that time it will show on the telnet side again.

Comments, questions, problems, suggestion? Feel free to continue the conversation here.

73 & good DX'ing!

andyz - K1RA

[Josh Logan]

I tried this out.  I was able to get RXClus to work, but SpotCollector still sees it as not working.

I don't know if it's the -# in the line, or if the initial handshake does not quite complete.

I really appreciate the work to get a good filter program working against the spots.  It will make it easier to know which band to focus on first.

Below is the SpotCollector line difference:

DX de KD7HGL-#:  14074.0  KG6OW        FT8    3 dB DE 14075287 Hz         2011Z
DX de KD7HGL-#:  14074.0  KF7MTE       FT8    0 dB DE 14076268 Hz         2011Z
DX de KD7HGL-#:  14074.0  KA2DUT       FT8  -16 dB DE 14074694 Hz         2011Z

N6WS dx cluster:

login: Hello KD7HGL, this is N6WS-6 in Nipomo, CA
running DXSpider V1.57 build 338
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-------------------------------------------------------------------------------
Date        Hour   SFI   A   K Forecast                              Logger
21-Nov-2020   18    82   8   3 No Storms -> No Storms                <W0MU>
-------------------------------------------------------------------------------
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Nodes: 21/440 Clr - Users: 147/6482 Clr Max: 161/8314 Clr - Uptime:  47d 15h 2m
Please enter your name, set/name <your name>
Please enter your QTH, set/qth <your qth>
Please enter your location with set/location or set/qra
Please enter your Home Node, set/homenode <your home DX Cluster>
KD7HGL de N6WS-6 21-Nov-2020 2012Z dxspider >
DX de HA5PP:       7039.6 P3AA                                        2012Z
DX de DL1NCH:      3573.0 OZ8BLR       FT8 -05dB from JO46 814Hz      2012Z
DX de DL1NCH:      3573.0 OZ8BLR       FT8 -05dB from JO46 814Hz      2012Z
DX de HG6Y:        3538.0 EW8DX                                       2013Z
DX de SV1YH:       7047.5 UX1MW        FT4 -06dB from KN98 2038Hz     2012Z

I'll have to read up on RXClus and see if that would also be useful.

73, KD7HGL

Josh

--
You received this message because you are subscribed to the Google Groups "SparkSDR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sparksdr+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/c3d806c4-0db9-4dc2-959c-65369420535an%40googlegroups.com.

[Andy Zwirko]

Hi Josh

  Thanks for testing out my new SparkSDR2DXC app.  I've taken a closer look at the output and I've done a few things that will hopefully help it work with SpotCollector.  For one, I fake a login, by printing out "login:" when a telnet client first connects.  I don't care or use anything you might send, but some client apps may be searching for an indication that a successful connection was made to the server.  Secondly I print a prompt line ending in a ">" which may also be used as an indication of successful login.  Finally I noted the line format was off by a few extra spaces, so I did a better job of aligning the columns, just in case.

  Please note that I print a version # when you start my app, or connect via telnet, so in the future one can easily let me know what you're running if we need to debug any further.  The latest SparkSDR2DXC.exe app can be downloaded from here:

https://drive.google.com/file/d/1xhoOb7ISHz1IZG9hemlVf7O67FKalVMK/view?usp=sharing

Please let me know any successes or issues you may have and thanks for any other feedback you can provide.  

73

andyz - K1RA

[Josh Logan]

It's almost working, but the prompt needs to be changed a little bit...  I don't fully understand, but I made a Telnet server and was able to find what seems to work.

If SpostCollector has Username as KD7HGL the prompt needs to be like:

KD7HGL de sparkSDR >

You send:

de KD7HGL 22-Nov-2020 0737Z SparkSDR2DXC >

I tried blank and space in SpotCollector, but nothing matched.

If you set the username to something static we can use any value in SpotCollector.  So you do not need to be call sign aware as long as it matches.

As an example:

Username = CLUSTER

The prompt needs to match:

CLUSTER de SparkSDR >

If there is any documentation about this I could not find it.  The most common software seems to be http://www.dxcluster.org/ 

Let me know when you have another version to try out.

Thanks!

73, KD7HGL

Josh

[Andy Zwirko]

OK Josh, I updated my app (v0.0.9) so the call/string SpotCollector should look for is now CLIENT.  Here's what it looks like when you connect on the telnet side;

Welcome to SparkSDR2DXC version 0.0.9 by K1RA

login:

CLIENT de SparkSDR2DXC >
DX de K1RA-#:    24919.0  TR8CA          FT4  -11 dB CQ 1050 Hz JJ40  1507Z
DX de K1RA-#:    14080.0  ON4CBK         FT4   -2 dB DE 2474 Hz       1507Z
DX de K1RA-#:    28074.0  PY2DOR         FT8  -17 dB DE  956 Hz GG66  1507Z
DX de K1RA-#:    18100.0  AF6JO          FT8   -5 dB CQ 1222 Hz DM12  1507Z

Let me know if that works for you.

Latest version can be downloaded from:

https://drive.google.com/file/d/1xhoOb7ISHz1IZG9hemlVf7O67FKalVMK/view?usp=sharing

73

andyz - K1RA

[Josh Logan]

Working great!

Thanks for the updates!

73, KD7HGL

Josh

[Andy Zwirko]

Josh,

Glad to hear it! And good to see it working.

I'm now starting to work on another app that interfaces SparkSDR Websockets to apps that can ingest the WSJTX-UDP protocol like GridTracker

https://gridtracker.org/

JTAlert and others.

More to follow.

andyz - K1RA

[ahop...@googlemail.com]

Hi Andyz,

nice work, it is great to see the things everyone is doing with the websocket interface.

73 Alan M0NNB

[Josh Logan]

I use JTAlert and gridtracker as well and will be happy to give it a try.  Do you know how those applications deal with multiple bands at once?  

That is why I was glad to see SpotCollector working, that is built to work with band data from the get go.  Now to learn how it works.

73, KD7HGL

Josh

[Andy Zwirko]

Josh

  You can see my SparkSDR2WSJTX-UDP app and GridTracker working in conjunction with SparkSDR 2.0.4.2 in a short 3 minute video I posted here

https://youtu.be/KgDrp8VlRik 

It's a screen capture video using Camtasia and not as smooth as running locally, but you'll get the idea.

I'll start another thread here shortly with a link to download the app so you can test it out too.  I don't run JTAlert, but I know it uses the same WSJT-X UDP protocol, so I'd be interested to know how it handles multiple, simultaneous band spot data.  GridTracker allows one to plot spots on the map by auto tracking by active band, mix all bands together, or filter by a band, mode, or combination of both and seems to work OK.  I prefer this over PSKreporter as it is local and instantaneous.

73

andyz - K1RA

[Josh Logan]

Awesome, thanks! 

That will be great to have running!

73, KD7HGL

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/CAL%3DH6pyvtHZmhYG0c_19h9VXXug02t4T4E-9J7rGT7WRfXkorw%40mail.gmail.com.

[S.ANICHINI]

Hello Andy

I am running  both on my skimmer computer (i just add a big FAN :-)) very nice !!!

Tks again for sharing

Do you use PERL again ?

73 Sylvain F1EYG

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/CAPaLT3ZkdTZJwKkRHSwo3Xbp0z4vwMbKHmiuwDp1JKUyg34wjA%40mail.gmail.com.

[IV3DXW Claudio]

Hello Andy,

I tested sparksdr2dxc and it works fine!

Thanks

73

Claudio iv3dxw

[IV3DXW Claudio]

Hello Andy,

if I save spots with PuTTY, I have two line-spaces each data row.

May be possible to avoid this?

Thanks

73 Claudio iv3dxw

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2020.12.24 23:51:46 =~=~=~=~=~=~=~=~=~=~=~=

Welcome to SparkSDR2DXC version 0.0.9 by K1RA


login:


CLIENT de SparkSDR2DXC >

DX de IV3DXW-#:  10136.0  K0LUZ          FT8   -5 dB DE 1261 Hz EL98  2251Z


DX de IV3DXW-#:   7074.0  F4IGV          FT8   11 dB DE 1499 Hz IN98  2251Z


DX de IV3DXW-#:   7074.0  IK0EIA         FT8  -13 dB DE  847 Hz       2251Z


DX de IV3DXW-#:   7074.0  F4FLF          FT8  -11 dB DE 1499 Hz JN18  2251Z

[AndyZ K1RA]

A new version of my SparkSDR2DXC DXCluster telnet server for SparkSDR is available here

https://drive.google.com/file/d/1xhoOb7ISHz1IZG9hemlVf7O67FKalVMK/view?usp=sharing 

I've checked with PuTTY and do not see any extra line breaks.

I've also added another optional command line argument to allow remote PC's to connect to the telnet server.  Program usage:

sparksdr2dxc MYCALL SPARKSDR_IP [THIS_PC_IP]

where

MYCALL = your callsign

SPARKSDR_IP = IP address of SparkSDR software (127.0.0.1 if running on local PC)

THIS_PC_IP = ethernet or WiFi private IP address (optional) - example: 192.168.0.100

If THIS_PC_IP is not supplied the telnet server listens on 127.0.0.1 (localhost) port 7373.  Use "ipconfig" at a "cmd" prompt to find your private IP.

You may have to adjust your Windows firewall settings to allow external connections from other PC's on your network.

[IV3DXW Claudio]

Hi Andy,

sparksdr2dxc MYCALL SPARKSDR_IP [THIS_PC_IP] is working fine. I can use this server from other PCs now!

I found an issue with telnet server and my Log4OM v2 logging program.

Not all the spots from SparkSDR are parsed.

I attach:

PuTTY log of IV3DXW  sparksdr2dxc local cluster (extra line breaks are present with default PuTTY config)

PuTTY log of IV3SCP web telnet cluster 

Log4OM V2 log  of spots collector and aggregator

I cannot manage at the moment special characters visualization.

I should investigate with Procomm terminal program (or equivalent).

Thanks

73 Claudio iv3dxw

[Andy Zwirko]

Claudio
  Regarding PuTTY.  How is your program configured for Settings > Terminal options?  Here are mine

Note the two "Implicit" options are both OFF.  If they are checked ON, the program will add additional characters to every line.  Maybe that is the problem?

Regarding missing spots, this may be true as compared with other spot sources as my program caches callsigns for up to 5 minutes before re-spotting them.  So, if my program first saw  W1XYZ four minutes ago from your SparkSDR and that station was seen again and again up until right now, it would not re-spot or send out the telnet port until the 5 minute cache expired for that callsign.  

Note also if you ran my DXC program on two different computers, starting at different times, but both pointing to the same SparkSDR server, their telnet spots would look slightly different based on the cache expiration timer too.  

I have the ability to adjust the cache expiration timer in my SparkSDR2WSJTX-UDP app and given your feedback, I see some people may want to adjust on the cache timer on the DXC program. I've added a new option and a few new features to my latest version (v0.1.3) for download here:

https://drive.google.com/file/d/1xhoOb7ISHz1IZG9hemlVf7O67FKalVMK/view?usp=sharing

Usage as follows

sparksdr2dxc MYCALL SPARKSDR_IP [THIS_PC_IP] [CACHETIME]

where

MYCALL = your callsign
SPARKSDR_IP = IP address of SparkSDR software (127.0.0.1 if running on local PC)
THIS_PC_IP = ethernet or WiFi private IP address (optional) - example: 192.168.0.100

CACHETIME = (optional) timeout in minutes to expire spotted calls in database (default: 5 or >=1)

Note to use and define CACHETIME you must also define THIS_PC_IP as either 127.0.0.1 or your private IP.

Also now if you reconnect to the DXC server I automatically expire the database cache.  This will allow you to immediate starting seeing every spot from SparkSDR right away, rather than waiting for the internal timer to do its job.  

Finally, I've made one more change to this new version of my program on how spots are processed from SparkSDR.  I now utilize the "msg" JSON field from the WebSocket to extract the call and grid, rather than use the individual 'call' and 'grid' JSON fields that are provided by SparkSDR.  There seems to be some cases where those are blank or undefined.  This will hopefully ensure all/more packets are decoded and sent to the telnet clients.

Please continue to provide feedback.  I appreciate it.

Thanks & 73

andyz - K1RA

You received this message because you are subscribed to a topic in the Google Groups "SparkSDR" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sparksdr/SgclcMcthaY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sparksdr+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/4b8027df-b1ca-4d3a-98f9-b0246cb90580n%40googlegroups.com.

[IV3DXW Claudio]

Hi Andy,

I tested latest version with this settings and it works fine!

C:\RADIO\sparksdr2dxc.exe IV3DXW 192.168.0.86 192.168.0.196 5

192.168.0.86 ethernet PC<>HL2

192.168.0.196 wifi

5 min cache

I see callsign spottred again after 5min if same band and mode. Good!

Now the output is clean, no blank callsign are present.

I will investigate if Log4OM 2 does miss some important DX and&or I will ask help in its forum.

Also PuTTY is not saving correctly even with settings you suggest, I will try another terminal program.

Just one question. In case of two users, the second login will reset the cache also to the first user?

73

Claudio IV3DXW

[Andy Zwirko]

Claudio  

  Glad it's working for you.  I don't save the PuTTY log to a file, so I will try that and see what happens here.

  Regarding your question, yes anytime a connection is made to the server it will reset the cache for all connected clients.  I never expected this system to be used in a public, multiple user environment, so I figured this type of reset wouldn't be a big problem.  If I was to rewrite this for multi-user, public support I'd probably put more effort into making this a bit more robust.

73

andyz - K1RA

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/0ba3b98e-0159-41bc-b5b5-5d23e8c39d99n%40googlegroups.com.

[Juergen, DL8LE]

Hello, I am using Wintelnet-X to summarize different telnet clusters and this works fine with N1MM (did not try it with Spotcollector). I am using the Hermes L II together with SparkSDR. At the moment spots are displayed for 80 and 40 m. Now I am stuck somehow with some questions:

1. Is there an easier way to start SparkSDR2DXC than going via the cmd command? Not important, would just be a little bit more convenient.

2. I don't know how to get more receivers in SparkSDR. My original idea was to have 5 instances running on 80,40,20,15,10 in FT8 and the same in FT4, i.e. in total 10. My HL Software informare from SparkSDR shows Firmware Version 72, Firmware Patch 8, Board ID 5 and Receivers 4.

3. Is there a way to extend the bandwith of one of those receivers from 3 (actually displayed in SparkSDR) to 10 khz? The HL II should be able to do this, but I am not sure.

I remember there were discussions about up to 8 or 10 instances of WSJT-X which could be connected to SparkSDR2DXC but I am unable to find the information again.

Any type of help would be very nice.

73

Juergen, DL8LE

[Juergen, DL8LE]

Additional info: after about 3 hours of working without any problems the program stopped with the error message "Can't use an undefined value as as ARRAY reference at script/sparksdr2dxc.pl line 116" Restart of the program was ok.

73

Juergen, DL8LE

[Josh Logan]

Please look at the CICRX variant.  This gateware can be used with SparkSDR to load 10 receivers:

• variants/hl2b3to4_cicrx - 10RX only gateware for Hermes-Lite 2.0 beta3 or beta4. This only supports 192kHz receivers. This uses only CIC filters and consequently only about 70kHz of the spectrum is usable. This is for multiband skimming.

https://github.com/softerhardware/Hermes-Lite2/tree/master/gateware/bitfiles/stable/20210730_73p2/variants/hl2b5up_cicrx

I'm currently using SparkSDR and sparksdr2dxc to feed spotcollector.

To make Sparksdr2dxc slightly more reliable I run it in a loop:

while ($true) {& '.\sparksdr2dxc.exe' KD7HGL 127.0.0.1 ; start-sleep -seconds 1}

73, KD7HGL

Josh

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/2a80f94e-0a0b-4802-9496-0470576f3e09n%40googlegroups.com.

[Juergen, DL8LE]

Thanks for the info, Josh.

I will look at the 10RX only gateware after I have had some success in getting the spots shown in the cmd window as well as in Spotcollector's telnet cluster window to be dispalyed in SC's database window, i.e. the main window, as well. This doesn't happen at the moment. As you this is working for you according to your mail I am asking you now what could be the possible reason. Dave, AA6YQ, author of DXLab could not provide a solution so far except to take out all filters which I did already (even though he is very supportive as you know).

The main difference between the spots from SparkSDR2DXC and another source (our club station rbn.dr4w.de) in Spotcollector's telnet cluster windows is that for some unknown reasons there is a blank line between the spots sent by SparkSDR2DXC and there are no blank lines by those sent from other sources. There are no blank lines shown in the cmd window and there are no blank lines either when I try to display the spots in N1MM+ telnet cluster and the bandmap. So I wonder what is causing those blank lines at the moment which l see is the only difference at the moment. ...

By the way: my Windows 10 pc did not accept your string in the cmd window. So something else I need to find a solution for. The array failures mentioned above are displayed every time when I change something at the settings of one of the receivers in SparkSDR, so not a big deal any more. Everything is running pretty well now for more than 24 hours without interruption.

It would be great if you could provide some help here.

73

Juergen

[Juergen, DL8LE]

Here is a screenshot to show the difference. Might be that it will help to better explain it. It is strange that the spot source status for SparkSDR2DXC is yellow and not green, at least an indication that something is wrong ...

73

Juergen, DL8LE

[Josh Logan]

That while ($true) loop needs to run under Powershell. Please give that a try.  I also believe this is the 2nd release of sparksdr2dxc.exe.

I can try to send that to you directly to see if that helps.

Spot Sources, here is my config:

You might try those values to see if that makes a difference.

I have blank lines just like you show.

73, KD7HGL

Josh

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/f15df565-ef96-4317-93e8-5e242b5ae90bn%40googlegroups.com.

[Juergen, DL8LE]

Hello Josh,

it's working now ... the clue to solve the problem was to compare your username with mine in SC's config. I had used DL8LE (as for all my other telnet clusters) and not CLIENT as it is given in SparkSDR2DXC as standard username. Foolish mistake ... But it's interesting that N1MM doesn't consider the username for the bandmaps but SC does.

The localhost as address doesn't work for me. There is an error message "Winsock error ...forcefully rejected ...". That's the same as I had before. But when I use the local ip address of my pc then everything is fine. SparkSDR, SparkSDR2DXC and DXLab are all running on the same pc, but it looks that in the latest version of SparkSDR2DXC I had downloaded it seems to be a must to use the local ip address (in N1MM the local ip has to be specified as well, just localhost or 127.0.0.1 doesn't work).

Thanks a lot for your help!

73

Juergen, DL8LE

[PE3ES / F4VTQ]

Works very good till this happens:

 18104.6  DP0GVN         WSPR -24 dB 37 1450 Hz IB59  1742Z
 18104.6  5T5PA          WSPR -17 dB 23 1457 Hz IL10  1752Z
 18104.6  DP0GVN         WSPR -20 dB 37 1449 Hz IB59  1812Z
 18104.6  5T5PA          WSPR -21 dB 23 1462 Hz IL10  1812Z
Can't use an undefined value as an ARRAY reference at script/sparksdr2wsjtx-udp.pl line 194.

This is with a SparkSDR Beta with SDRplay RSP1a on Windows 10

Op dinsdag 2 november 2021 om 20:14:20 UTC+1 schreef Juergen, DL8LE:

[Josh Logan]

I have seen that as well.  I use the following in PowerShell: 

# Loop, since it likes to crash:

while ($true) {& '.\sparksdr2dxc.exe' KD7HGL 127.0.0.1 ; start-sleep -seconds 1}

Replace with your callsign so it will just restart the process if it fails.

73, KD7HGL

Josh

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/4476850c-cf37-4248-90e7-68a41c0a1ddfn%40googlegroups.com.

[AndyZ K1RA]

For those who've used my SparkSDR2DXC and SparkSDR2WSJTX-UDP (PERL) apps, I've finally had the time to start revisiting and rewriting them (in Python).  I've currently got a new SparkSDR2DXC  v2  app running and will package and share it for distribution over the next few days .  As a recap the first app creates a local DXCluster like telnet server and generates formatted messages for unique spots seen by SparkSDR's FT8, FT4 and WSPR band decoders.  These messages can by digested by RXClus and other Telnet Clients.  The second app, which I've yet to rewrite, emulates WSJT-X's UDP broadcaster capability and allow apps like GridTracker to digest SparkSDR spots and plot them locally on a real-time map.  I hope to support mulitcast in v2 which should allow multiple apps like JTAlert and GridTracker to monitor SparkSDR simultaneously.

I've now got several fixes with the SparkSDR2DXC v2 allowing the handling of WebSocket reconnects if SparkSDR is rebooted/stopped, which was mentioned as an earlier v1 issue, as well as catching and gracefully handling SparkSDR WebSocket messages other than "spots", like band and mode changes so the app continues generating spots and doesn't die/exit.

I hope to package an EXE version for Windows 10 users and I will distribute the Python source for those wishing to run this under Python and/or Linux. I look forward to any future feedback on its usability.

73 and more to follow.

andyz - K1RA

[RD Powers]

I'm looking forward to a release of the rewritten 2nd app.  I am currently using SparkSDR with a HL2 radio to skim four bands at once.  I'd like to be able to view the results using GridTracker.  I am interested in the standalone Python code for Linux.  I know, that's a lot to ask for, so I will be patient.

[AndyZ K1RA]

For release SparkSDR2DXC v2.00, a DXCluster Telnet server for SparkSDR FT8, FT4 and WSPR spots received over Websockets. Download from my Google Drive.  

https://drive.google.com/file/d/1-JTOw-plVbQ6KAhbRGONlGGVjO1mo4pb/view?usp=sharing

This new version is written in Python and the downloaded ZIP contains both a standalone executable tested on Windows 10, as well as the Python source code and requirements.txt file. The later could be used with the Python interpreter and should run under Linux or MacOSX.  I've not tried that on my Ubuntu system, so I'm looking for any feedback for those who try using the source code directly.

./sparksdr2dxc.exe -h                  
Usage: sparksdr2dxc.py -m MYCALL [-c cache_expire] [-s sparksdr_ip] [-t telnet_port] [-d] [-v] [-h]
-m MYCALL (mandatory, ex: -m K1RA)
-c cache_expire (optional, minutes, ex: -c 5) [default=5]
-s sparksdr_ip (optional, ip address, ex: -s 192.168.1.234) [default=127.0.0.1]
-t telnet_port (optional, tcp port, ex: -t 2323) [default=7373]
-d (turns on debug messages)
-v (prints version)
-h (prints this help)

The only mandatory command line option in -M followed by your call.  -d is useful to show Websocket data and decoded messages sent to Telnet clients.  Once a call is spotted on a mode's base frequency (kHz) it will not be resent out the Telnet server until the cache timer (5 minutes default) has expired for that hashed call+frequency.  

By default it is assumed the app and your SparkSDR are running on the same computer, if not, use the -s option to point to the SparkSDR's remote IP on your LAN.

The default Telnet port to connect to with your client is 7373, use the -t option to change that if its already in use.

If this appears to work for people, and/or once I work through any bugs/feedback I will then convert my SparkSDR2WSJTX-UDP over to Python and publish a new version for Windows 10 and source code for Linux.

Ctrl-C should exit the program.

>telnet localhost 7373

login:
Welcome, you are client 0.
DX de   K1RA-#:  14080.0  VP8LP          FT4     6 dB DE 2069 Hz        2346Z
DX de   K1RA-#:  14080.0  W9KEY          FT4    -7 dB DE  591 Hz        2346Z
DX de   K1RA-#:  14080.0  EA5HM          FT4   -14 dB CQ  784 Hz IM99   2346Z
DX de   K1RA-#:  18100.0  W5ZPM          FT8     8 dB DE 1584 Hz        2346Z
DX de   K1RA-#:  18100.0  TY0RU          FT8    -5 dB DE  943 Hz        2346Z
DX de   K1RA-#:  18100.0  N5FWB          FT8    17 dB DE 1267 Hz EL29   2346Z
DX de   K1RA-#:  18100.0  W4DAN          FT8    -7 dB DE  633 Hz        2346Z
DX de   K1RA-#:  18100.0  KO4BEE         FT8    -6 dB CQ 1635 Hz FM05   2346Z
DX de   K1RA-#:  18100.0  KC8GCR         FT8   -19 dB DE 2308 Hz EM95   2346Z
DX de   K1RA-#:  21074.0  K5KXF          FT8    16 dB DE 1344 Hz        2346Z
DX de   K1RA-#:  21074.0  KG6PH          FT8    11 dB CQ 2716 Hz EM13   2346Z
DX de   K1RA-#:  21074.0  VE7DH          FT8     7 dB CQ 2626 Hz CN89   2346Z
DX de   K1RA-#:  21074.0  KE7RTB         FT8     5 dB DE 1215 Hz        2346Z
DX de   K1RA-#:  21074.0  W5ATG          FT8    -9 dB DE 1631 Hz EL09   2346Z
DX de   K1RA-#:  21074.0  VE7GPJ         FT8    -4 dB DE 1101 Hz        2346Z
DX de   K1RA-#:  21074.0  KD6DJ          FT8     4 dB DE 2332 Hz        2346Z
DX de   K1RA-#:  21074.0  KB5WSJ         FT8     0 dB CQ 2033 Hz EM10   2346Z
DX de   K1RA-#:  21074.0  NA7KR          FT8    -4 dB DE  965 Hz        2346Z
DX de   K1RA-#:  21074.0  AD8J           FT8    -5 dB CQ 1463 Hz EM85   2346Z
DX de   K1RA-#:  21074.0  KI5NVD         FT8     0 dB DE  923 Hz        2346Z
DX de   K1RA-#:  21074.0  KA9ONW         FT8   -16 dB DE 2196 Hz        2346Z
DX de   K1RA-#:  21074.0  KD9IDD         FT8    -8 dB DE 2344 Hz EN52   2346Z
DX de   K1RA-#:  21074.0  W4JVN          FT8    -1 dB CQ 1384 Hz FM17   2346Z
DX de   K1RA-#:  21074.0  WA3OGQ         FT8   -11 dB DE 1451 Hz FM09   2346Z
DX de   K1RA-#:  21074.0  W7MY           FT8   -17 dB DE 2987 Hz        2346Z
DX de   K1RA-#:  21074.0  W9OAA          FT8   -13 dB CQ 1060 Hz EM69   2346Z

DX de   K1RA-#:  18104.6  AI7HP/2       WSPR   -18 dB   23 dBm   CN87   2346Z
DX de   K1RA-#:  18104.6  KB3LC         WSPR   -24 dB   33 dBm   FM19   2346Z
DX de   K1RA-#:  21094.6  K6SGW         WSPR   -20 dB   23 dBm   CM87   2346Z
DX de   K1RA-#:  18100.0  LW5DIX         FT8   -10 dB DE 1142 Hz GF05   2347Z
DX de   K1RA-#:  21074.0  KC4OQB         FT8     2 dB CQ 2872 Hz EM13   2347Z
DX de   K1RA-#:  21074.0  VA7VOC         FT8     0 dB DE  929 Hz DN09   2347Z
DX de   K1RA-#:  14095.6  VE6CLE        WSPR   -21 dB   20 dBm   DO33   2346Z
DX de   K1RA-#:  14095.6  N0BKB         WSPR   -13 dB   23 dBm   EN21   2346Z
DX de   K1RA-#:  14095.6  K6MCS         WSPR     6 dB   37 dBm   CM98   2346Z
DX de   K1RA-#:  14095.6  NJ6E          WSPR   -17 dB   23 dBm   CM88   2346Z

Comments, feedback, suggestions appreciated.

73

andyz - K1RA

[AndyZ K1RA]

I've just been informed by Google that EXE files are treated as viruses, so I have currently placed the ZIP file for download here until they review my request to host on Google Drive.

https://k1ra.us/wp-content/uploads/sites/5/download/sparksdr2dxc-v2.00.zip

Please let me know if you have any issues accessing it.

andyz - K1RA

[Josh Logan]

I needed one small patch for SpotCollector to work:

                # Send a welcome message
                tserver.send_message(new_client, "login: ")
                tserver.send_message(new_client, "Welcome, you are client {}.".format(new_client))
+              tserver.send_message(new_client, "CLIENT de SoarkSDR2DXC")

Thanks for the update and python code, that was nice to be able to look to see how I could fix it.

73, KD7HGL

Josh

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/f11de7af-c907-4404-9628-e652022f0bc8n%40googlegroups.com.

[AndyZ K1RA]

I've made some minor improvements to my SparkSDR2DXC DXCluster server app.  This includes a request from Josh KD7HGL to allow it to work with SpotCollector, as well as some more stringent call sign checking on spots that come out of the FT8, FT4 and WSPR decoders.  Also, if you start the app on the command line with the -d option you'll see debug output including Websocket spots from SparkSDR, their conversion to DXCluster telnet lines, as well as any spots I flag as invalid or not matching typical call sign and report formats.  Use the -h option to get help on other command line options.

> sparksdr2dxc.exe -m K1RA -d -s 192.168.2.148
sparkdxc2dxc version 2.0.1 by K1RA
Opening Connection
Message Received: {"cmd":"spotResponse","spots":[{"time":"2022-10-23T21:27:45Z","frequency":10138729.0,"tunedfrequency":10136000.0,"power":0,"drift":0,"snr":13,"dt":0.1,"msg":"CQ WC4J FM18","mode":
"FT8","submode":"","distance":73.30486811593546,"call":"WC4J","color":1,"locator":"FM18","valid":true,"offsetFrequency":2729.0,"rxid":16}, ...

DX de   K1RA-#:  10136.0  WC4J           FT8  13 dB CQ 2729 Hz FM18     2127Z
DX de   K1RA-#:  10136.0  WZ8DX          FT8  16 dB DE 1368 Hz EM79     2127Z
<<<--- INVALID --->>> 212700  10138.7 WSPR       Q3QQQ 1455  -9 EM10  

Download version 2.0.1 here:

https://k1ra.us/wp-content/uploads/sites/5/download/sparksdr2dxc-v2.0.1.zip

Please let me know if you have any issues and comments, questions, feedback welcome.

73

andyz - K1RA

[RD Powers]

I was able to get the code to work on Linux, Debian 11 (bullsye) variant. 

The only problem I encountered was a local package installation issue and the following advice sorted it out for me:
https://stackoverflow.com/questions/48730108/attributeerror-module-websocket-has-no-attribute-websocketapp-pip 

It is working flawlessly now.

I look forward to when/if you port the websocket-to-wsjt-x code!

I'd love to pump the data from sparksdr into something like GridTracker, but it doesn't seem to have an option to connect to a telnet server.

I haven't found a good Linux app for DX Clusters, if anyone know of one please let me know.

Regards,

RDP

[AndyZ K1RA]

RDP

  Thanks for the feedback and I'll try to add the extra package installation advice to a Linux README.

  I did update my SparkSDR2WSJTX software back in Oct 2022, you can download here

https://k1ra.us/wp-content/uploads/sites/5/download/sparksdr2wsjtx-udp-v2.0.0.zip

and read comments and others user experiences here

https://groups.google.com/g/sparksdr/c/AJnF74z5Sgk/m/ydK17RyBAQAJ

73

andyz - K1RA

[IV3DXW Claudio]

Hi Andy,

I have used first version of SparkSDR2DXC for long time, very nice tool!

Today I tried version 2.0.1 because I was not able  to connect from another PC to SparkSDR2DXC even if IP was correctly set.

Version 2.0.1 is OK, I can connect inside my home network.

But I found a (possible?) bug: the string has different spacing/tabbing after FT8 and before TIME

DX de IV3DXW-#:  18100.0  JA3KVT         FT8  -16 dB DE  755 Hz       0932Z (old ver)

DX de IV3DXW-#:  21074.0  PA3BUD         FT8 -10 dB DE 1019 Hz          0933Z (new ver)

The old version format was compatible with SKIMMER output so the parsing was OK with applications connected to SparkSDR2DXC.

But with the new ver the parsing of FT8 spot is not working.

Thanks

73 Claudio iv3dxw

[AndyZ K1RA]

Hi Claudio,

  Thanks for the feedback.  I telnet'ed into telnet.reversebeacon.net port 7001 DXcluster server and checked their SKIMMER output format 

DX de WC2L-#:    24915.0  PY2RND         FT8  -20 dB  GG68    CQ      1644Z
DX de KL7RA-#:    7074.0  WH6CLZ         FT8  -14 dB          CQ      1644Z
DX de R6YY-#:    18100.0  OH1NDA         FT8   29 dB          CQ      1644Z
DX de VE6AO-#:   21074.0  IK4LZH         FT8  -18 dB  JN54    CQ      1644Z
DX de SM7IUN-#:  21074.0  BG0CAB         FT8   -4 dB  NN33    CQ      1644Z
DX de VU2CPL-#:  21074.0  OE1TRB         FT8  -18 dB  JN88    CQ      1644Z
DX de WA7LNW-#:  21140.0  F8MRQ          FT4   -9 dB          CQ      1644Z
DX de KH6LC-#:   14074.0  YH5UG          FT8  -15 dB  OJ11    CQ      1645Z
DX de JI1HFJ-#:  10136.0  KL7RRC         FT8  -11 dB  AO63    CQ      1645Z
DX de VY0ERC-#:  21074.0  YH2UF          FT8  -19 dB  OI52    CQ      1645Z
DX de W1NT-6-#:  21140.0  F4CTJ          FT4   -6 dB  JN09    CQ      1645Z
DX de BA6KC-#:   21074.0  UA6LQZ         FT8   -6 dB          CQ      1645Z
DX de KM3T-3-#:  18100.0  G0VLF          FT8   -9 dB  IO94    CQ      1645Z

And then I updated my software to reflect the correct spacing.  Note I additionally provide TONE frequency of FTn stations and dBm for WSPR, so the my message portion of the DX line is slightly different.  Here is the new output from my SparkSDR2DXC v2.0.3 app.

DX de   K1RA-#:  50313.0  K1HTV          FT8   39 dB FM18 1908 Hz DE  1643Z
DX de   K1RA-#:  14095.6  W4WLO         WSPR  -25 dB EM50   30 dBm    1644Z
DX de   K1RA-#:  14095.6  W1/AJ8S       WSPR  -24 dB        33 dBm    1644Z
DX de   K1RA-#:  18104.0  G4YXG          FT4  -15 dB JO01  795 Hz CQ  1645Z
DX de   K1RA-#:  21140.0  HA8RK          FT4  -13 dB JN96 1165 Hz DE  1645Z
DX de   K1RA-#:   7078.0  KB8UVN         JS8    8 dB       962 Hz DE  1645Z
DX de   K1RA-#:  18100.0  RV3LE          FT8  -19 dB       615 Hz DE  1646Z

Please download the latest SparkSDR2DXC app version 2.0.3 from:

https://k1ra.us/wp-content/uploads/sites/5/download/sparksdr2dxc-v2.0.3.zip 

and let me know if you have any other issues.

For any new users please refer to this link earlier in this thread for starting this DXCluster command line app with its command line options:

https://groups.google.com/g/sparksdr/c/SgclcMcthaY/m/UN68K5kwDwAJ 

73

andyz - K1RA

[IV3DXW Claudio]

Hi Andy

Great! now is working fine, thanks!

I use RXCLUS program to collect spots and filter them.

Best 73, Claudio iv3dxw

[Grant R]

Has anyone managed to get telnet spots from sparksdr2dxc working in Log4OM2? I can see the spots arriving in the "cluster management window/setup" but not appearing in the Log4OM2 main screen cluster view. At first glance that suggests a filtering issue but I've explored that with no luck. I have other telnet connections that work. Most likely this is a question to ask on the Log4OM forum but a quick search there does not show any posts about sparksdr. Thought it worth asking here first.

Log4OM2 ver 2.30.1.5

SparkSDR2DXC ver 2.0.3

Windows 11

73

Grant VK2LX

[Andy Zwirko]

Grant

  Can you send me a text cut/paste of what a Log4OM telnet session looks like when connected to a working DXcluster?  The last update I performed on my SparkSDR2DXC add-on app to SparkSDR fixed an issue with RXclus not ingesting spots due to an extra space character in one of the fields.  Log4OM may be even more strict in field lengths.

Thanks & 73

andyz - K1RA

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/98465e18-896c-475c-8dc8-b67492d93452n%40googlegroups.com.

[AndyZ K1RA]

Thanks to Grant VK2LX I was able to work through several issues with my apps working in conjunction with Log4OM and Win4Yaesu.  

The new versions can be found at the links below:

SparkSDR2DXC (Telnet server for RXClus, WinTelnetX, Log4OM, Win4Yaesu, etc.)
https://k1ra.us/wp-content/uploads/sites/5/download/sparksdr2dxc-v2.0.8.zip

SparkSDR2WSJTX-UDP (UDP multicast for GridTracker)
https://k1ra.us/wp-content/uploads/sites/5/download/sparksdr2wsjtx-udp-v2.0.4.zip

73

andyz - K1RA

[Malcolm Warwick]

Hi Andy

Great pieces of software :)

I have successfully  installed both apps on Windows and am now trying to install on Linux server but am getting an error thus:

sysop@debian11:~/sparksdr2dxc$ python3 sparksdr2dxc.py -m vk2ski -s 192.168.1.180 -t 7300 -d
Traceback (most recent call last):
  File "/home/sysop/sparksdr2dxc/sparksdr2dxc.py", line 23, in <module>
    from websocket import WebSocketApp
ImportError: cannot import name 'WebSocketApp' from 'websocket' (/home/sysop/.local/lib/python3.9/site-packages/websocket/__init__.py)
sysop@debian11:~/sparksdr2dxc$

Are you able to assist please?

73

Malcolm -VK2SKI

[Andy Zwirko]

Hi Malcolm

  In the zip file should be a requirements.txt file.  You'll need that along with the Python interpreter and the 'pip' command to pull in all the libraries I use.  See this link on advice

https://note.nkmk.me/en/python-pip-install-requirements/ 

but basically

pip install -r requirements.txt

Good luck & 73

andyz - K1RA

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/b0bf5475-2264-4baf-be3b-bd26b77152een%40googlegroups.com.

[Josh Logan]

I'm trying 2.0.8 with SparkSDR 2.0.985 and I get this error:

PS C:\Users\Logan\Downloads\sparksdr2dxc-v2.0.8> python .\sparksdr2dxc-2.0.8.py -m KD7HGL  -d
sparkdxc2dxc version 2.0.8 by K1RA
Opening Connection
Message Received: {"cmd":"spotResponse","spots":[{"time":"2024-03-20T22:10:00Z","frequency":21079844,"tunedfrequency":21076000,"power":0,"drift":0,"snr":-30,"dt":-0.6,"msg":"K1ABJ YK0FZS NA56","mode":"JT65","submode":"","distance":15897.283014053643,"call":"YK0FZS","locator":"NA56","rxid":6}]}

Traceback (most recent call last):

  File "C:\Users\Logan\Downloads\sparksdr2dxc-v2.0.8\sparksdr2dxc-2.0.8.py", line 443, in <module>
    valid, hhmmss, freq, mode, call, grid, tone, snr, dt, dbm, msg, jtmsg = parse_message(json_msg)
                                                                            ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Logan\Downloads\sparksdr2dxc-v2.0.8\sparksdr2dxc-2.0.8.py", line 126, in parse_message
    tone = int(json_message["offsetFrequency"])
               ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'offsetFrequency'
Message Received: {"cmd":"spotResponse","spots":[{"time":"2024-03-20T22:10:52.5Z","frequency":18105814,"tunedfrequency":18104000,"power":0,"drift":0,"snr":-17,"dt":-0.1,"msg":"CQ WY0V EN12","mode":"FT4","submode":"","distance":2063.0038676305207,"call":"WY0V","locator":"EN12","rxid":18}]}
Message Received: {"cmd":"spotResponse","spots":[{"time":"2024-03-20T22:10:45Z","frequency":14074803,"tunedfrequency":14074000,"power":0,"drift":0,"snr":-5,"dt":0.1,"msg":"CQ W6/UT5UF","mode":"FT8","submode":"","distance":null,"call":"W6/UT5UF","locator":"","rxid":1},{"time":"2024-03-20T22:10:45Z","frequency":14076546,"tunedfrequency":14074000,"power":0,"drift":0,"snr":-12,"dt":0,"msg":"N8WH K7CTV DM42","mode":"FT8","submode":"","distance":1948.0661970177587,"call":"K7CTV","locator":"DM42","rxid":1},{"time":"2024-03-20T22:10:45Z","frequency":14076338,"tunedfrequency":14074000,"power":0,"drift":0,"snr":-14,"dt":0.1,"msg":"I0XNH KB9ZM EM57","mode":"FT8","submode":"","distance":2928.9555728587266,"call":"KB9ZM","locator":"EM57","rxid":1},{"time":"2024-03-20T22:10:45Z","frequency":14074520,"tunedfrequency":14074000,"power":0,"drift":0,"snr":-17,"dt":0,"msg":"K4AOQ WA7RCT R-23","mode":"FT8","submode":"","distance":null,"call":"WA7RCT","locator":"","rxid":1},{"time":"2024-03-20T22:10:45Z","frequency":14075645,"tunedfrequency":14074000,"power":0,"drift":0,"snr":-16,"dt":0.4,"msg":"V51WH W5LGA DM65","mode":"FT8","submode":"","distance":1861.3425646076726,"call":"W5LGA","locator":"DM65","rxid":1},{"time":"2024-03-20T22:10:45Z","frequency":14074272,"tunedfrequency":14074000,"power":0,"drift":0,"snr":-12,"dt":-0.7,"msg":"CQ N0XO DN35","mode":"FT8","submode":"","distance":757.9522789858239,"call":"N0XO","locator":"DN35","rxid":1}]}

Messages do keep scrolling.  I then try to have SpotCollectot connect, but I get:
Winsock Error # 10061 - Connection is forcefully rejected @ 3/20/2024 2213

I did upgrade these 2 lines to get the modules to install:

gevent==24.2.1
greenlet==3.0rc3

Any suggestions on how to get this working correctly?  I'm on WIndows 11, Python 3.11.5

73, KD7HGL

Josh

To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/CAL%3DH6pwrfnT7d2x1ciqt5NGNUsESxwOBqWDRZBcn0Kz988cGkg%40mail.gmail.com.

[AndyZ K1RA]

Hi Josh

  Yes this is an error occurs with the several of the latest BETA versions Alan has created since after SparkSDR 2.0.975 as noted in my post here on Jan 22:

https://groups.google.com/g/sparksdr/c/cOrnMBpiJhk/m/2iCAkaeiBwAJ 

  Something changed in his code and as I've not gotten a response yet from him, I'm reluctant to changed my code or develop and keep up against future BETA versions.

73

andyz - K1RA

[ahop...@googlemail.com]

Hi Andy,

sorry this I forgot about this, I'll have a good look.

73 Alan M0NNB 

[ahop...@googlemail.com]

Hi Andy,

It appears I did remove a few fields that were only meant to be used internally. I could put the offsetFrequency back but it is just frequency - tunedfrequency so is redundant and extra bandwidth.  My preference would be to not put it back, but could be persuaded.

73 Alan M0NNB

[AndyZ K1RA]

Hi Alan

  If it has been permanently removed, and not an oversight/mistake, then I will look into modifying my software.

73

andyz - K1RA

[ahop...@googlemail.com]

Hi Andy,

thanks and sorry for the change and my slow response.  I'll try not to change it again. The technical background was a change from using Newtonsoft json to serialize internal objects to json (internally using slow reflection) to using a very much faster code generation solution.  

73 Alan M0NNB