SparkSDR meets Federated Wiki

[Ward Cunningham]

Federated wiki was born as a decentralized data sharing platform nine years ago. Digital radio has renewed our interest in wiki’s data capability and especially the newly created WebSocket interface which mates well with wiki's client-side. Wiki pages “line up” and pass information left to right. Here is a picture of code written in the 24 hours since my son Pat got me hooked up with version 2.0.3.8 which opens this door. 

I’m including a screen shot which I hope makes it through to you. But if not, you can start here with your own SparkSDR running and browse these wiki pages:

http://ft8.ward.asia.wiki.org/spark-decodes.html

Expect more before the weekend is over. Thanks and best regards — Ward, K9OX 

On Nov 14, 2020, at 5:13 AM, spar...@googlegroups.com wrote:

spar...@googlegroups.com

Google Groups

Topic digest
View all topics

• Operating FT8 problems - 18 Updates

Operating FT8 problems

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 10:00AM -0800 Sorry Alan,   I was a bit light with information. I am using the built in FT8 mode in SparkSDR and running ver 2.0.3-5 with an HL2 code version 72. The system is Linux Mint 20 and I am running on my local lan with router and switches.   It seems that I am transmitting a corrupted code which only ZL1AMI can decode and as a consequence his RX seems to be corrupted in that he can only decode me.   Thanks for your interest,   Graeme ZL2APV   On Saturday, November 14, 2020 at 1:11:52 AM UTC+13 ahop...@googlemail.com wrote:

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 10:01AM -0800 Oh yes I should have mentioned that I can't decode ZL1AMI.   --G

"ahop...@googlemail.com" <ahop...@googlemail.com>: Nov 13 10:04AM -0800 Most strange, what software/radio is ZL1AMI using? Do you both have accurate clocks? 73 Alan M0NNB

"ahop...@googlemail.com" <ahop...@googlemail.com>: Nov 13 10:14AM -0800 It is also worth looking on pskreporter to see if anyone decoded you.

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 10:43AM -0800 It does not look like I have been decoded although I have decoded plenty of others on PSK reporter. Maybe I should try reverting to the latest stable version of SparkSDR? As I had no idea of how it works I made a few assumptions. SparkSDR is all I need and if it runs WSPR (which it does) it should run FT8 The callsign is automatically populated in the left hand set of message boxes and clicking on the icon initiates a CQ If I get a response it will appear in the station list maybe in another colour? If I click on a responding station it will populate my LHS messages with the messages to run the QSO It will automatically progress through the QSO messages or alternatively I may have to click on each one   As you can see, I am pretty green if I don't know the answers to the above so I'm not sure if I'm doing something wrong, my setup is wrong or there is a bug On Saturday, November 14, 2020 at 7:14:55 AM UTC+13 ahop...@googlemail.com wrote:

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 10:46AM -0800 I missed the clocks question yes Time.is reports exact time for both of us. ZL1AMI is using the same setup as me although I think his HL2 firmware is a version behind mine.

"ahop...@googlemail.com" <ahop...@googlemail.com>: Nov 13 10:56AM -0800 I don't think the spark or hl firmware versions will make a difference. It is particularly odd that ZL1AMI is not decoding anything else, the decoding side of Spark calls the wsjtx decoder directly so I can't get much wrong here. What versions of wsjtx are you both using? It is possible that my ft8 encoding is failing as more of that is done in spark and there could be problems with things changing in wsjtx versions. I shall test tx over the weekend. Maybe we have created a secure channel :) 73 Alan M0NNB

"ahop...@googlemail.com" <ahop...@googlemail.com>: Nov 13 10:59AM -0800 It is worth checking the log files and I have just thought of something that could have changed in the last spark release so trying an older version might be worth a go.

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 11:07AM -0800 I am using wsjtx version 2.1.2   I will try to find the log files and have a look and if I see nothing will try the latest stable version of SparkSDR   On Saturday, November 14, 2020 at 7:59:56 AM UTC+13 ahop...@googlemail.com wrote:

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 11:11AM -0800 This is a sidetrack but I did get a crash last night and here is the log for it 8/07/2020 1:14:32 pm v2.0.2.3 Main error The type initializer for 'radio.vtrx' threw an exception. SparkSDR Void loadSettings() at radio.wirelessModel.loadSettings() at radio.wirelessModel..ctor() at SparkSDR.App.OnFrameworkInitializationCompleted() at Avalonia.Controls.AppBuilderBase`1.Setup() at Avalonia.Controls.AppBuilderBase`1.SetupWithLifetime(IApplicationLifetime lifetime) at Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime[T](T builder, String[] args, ShutdownMode shutdownMode) at SparkSDR.Program.Main(String[] args) The type initializer for 'radio.sparkCore' threw an exception. SparkSDR T load_function[T](System.String) at radio.sparkCore.load_function[T](String name) at radio.vtrx.LoadFunctions() at radio.vtrx..cctor() Value cannot be null. (Parameter 'ptr') System.Private.CoreLib System.Delegate GetDelegateForFunctionPointer(IntPtr, System.Type) at System.Runtime.InteropServices.Marshal.GetDelegateForFunctionPointer(IntPtr ptr, Type t) at radio.sparkCore.load_function[T](String name) at radio.sparkCore..cctor() --

"ahop...@googlemail.com" <ahop...@googlemail.com>: Nov 13 11:18AM -0800 That log looks like it is from some time ago and probably when you installed without port audio being there, but I am interested that you had a crash, I think there is an Avalonia linux issue at the moment but have not been able to track it down, do you happen to know what you were doing at the time, were you moving the mouse or opening a dropdown and how long can I go in one sentence?

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 11:51AM -0800 Slipped off for some breakfast but back now. I opened the settings and it crashed spark. It has happened twice now.   On Saturday, November 14, 2020 at 8:18:48 AM UTC+13 ahop...@googlemail.com wrote:

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 11:54AM -0800 This may be an applicable log - Should I start another thread?   13/11/2020 11:48:57 am v2.0.3.5 Main error glXMakeContextCurrent failed Avalonia.X11 System.IDisposable MakeCurrent(IntPtr) at Avalonia.X11.Glx.GlxContext.MakeCurrent(IntPtr xid) at Avalonia.X11.Glx.GlxGlPlatformSurface.RenderTarget.BeginDraw() at Avalonia.Skia.GlRenderTarget.BeginRenderingSession() at Avalonia.Skia.SkiaGpuRenderTarget.CreateDrawingContext(IVisualBrushRenderer visualBrushRenderer) at Avalonia.Rendering.DeferredRenderer.EnsureDrawingContext(IDrawingContextImpl& context) at Avalonia.Rendering.DeferredRenderer.UpdateRenderLayersAndConsumeSceneIfNeeded(IDrawingContextImpl& context, Boolean recursiveCall) at Avalonia.Rendering.DeferredRenderer.Render(Boolean forceComposite) at Avalonia.Rendering.DeferredRenderer.Paint(Rect rect) at Avalonia.Controls.TopLevel.HandlePaint(Rect rect) at Avalonia.X11.X11Window.DoPaint() at Avalonia.X11.X11Window.<OnEventSync>b__104_0() at Avalonia.Threading.JobRunner.RunJobs(Nullable`1 priority) at Avalonia.X11.X11PlatformThreading.HandleX11(CancellationToken cancellationToken) at Avalonia.X11.X11PlatformThreading.RunLoop(CancellationToken cancellationToken) at Avalonia.Threading.Dispatcher.MainLoop(CancellationToken cancellationToken) at Avalonia.Controls.ApplicationLifetimes.ClassicDesktopStyleApplicationLifetime.Start(String[] args) at Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime[T](T builder, String[] args, ShutdownMode shutdownMode) at SparkSDR.Program.Main(String[] args) -- And this one   4/11/2020 3:01:37 pm v2.0.3.5 Main error glXMakeContextCurrent failed Avalonia.X11 System.IDisposable MakeCurrent(IntPtr) at Avalonia.X11.Glx.GlxContext.MakeCurrent(IntPtr xid) at Avalonia.X11.Glx.GlxGlPlatformSurface.RenderTarget.BeginDraw() at Avalonia.Skia.GlRenderTarget.BeginRenderingSession() at Avalonia.Skia.SkiaGpuRenderTarget.CreateDrawingContext(IVisualBrushRenderer visualBrushRenderer) at Avalonia.Rendering.DeferredRenderer.EnsureDrawingContext(IDrawingContextImpl& context) at Avalonia.Rendering.DeferredRenderer.UpdateRenderLayersAndConsumeSceneIfNeeded(IDrawingContextImpl& context, Boolean recursiveCall) at Avalonia.Rendering.DeferredRenderer.Render(Boolean forceComposite) at Avalonia.X11.X11Window.DoPaint() at Avalonia.Threading.JobRunner.RunJobs(Nullable`1 priority) at Avalonia.X11.X11PlatformThreading.HandleX11(CancellationToken cancellationToken) at Avalonia.X11.X11PlatformThreading.RunLoop(CancellationToken cancellationToken) at Avalonia.Threading.Dispatcher.MainLoop(CancellationToken cancellationToken) at Avalonia.Controls.ApplicationLifetimes.ClassicDesktopStyleApplicationLifetime.Start(String[] args) at Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime[T](T builder, String[] args, ShutdownMode shutdownMode) at SparkSDR.Program.Main(String[] args) --

"ahop...@googlemail.com" <ahop...@googlemail.com>: Nov 13 12:14PM -0800 Thanks, that is realy usefull. Did you notice if the settings started to slide out or was it after the settings were in full view?

Graeme Jury <gvj...@gmail.com>: Nov 14 09:21AM +1300 I had the settings displayed for a second or so and then crash

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 01:52PM -0800 Back to FT8. I have installed ver 2.0.1.9 and have had my first SparkSDR QSO with VK2SDX and it was all pretty intuitive and just how I expected it might have been. I am not sure if it is possible to capture the QSO for log purposes but I jotted it down anyway. I also installed the latest stable version which seemed to be transmitting but crashed each time I tried to respond to a call. I think I remember a post regarding a change to wsjtx affecting spark and a new non stable version was released so I was not too concerned. I will load the latest pre-release version again and have another go.   73, Graeme ZL2APV

"ahop...@googlemail.com" <ahop...@googlemail.com>: Nov 13 02:03PM -0800 OK, I'll give ft8 tx a good test over the weekend and try and find out what is going on. ZL1AMI's receive problem is still a mystery. 73 Alan M0NNB

"gvj...@gmail.com" <gvj...@gmail.com>: Nov 13 02:21PM -0800 Thanks Alan,   I can confirm that I have reinstalled Linux x64 SparkSDR.2.0.3-5.linux-x64.deb and it does not work on TX for me but all is well using 2.0.1.9 which I will stay with in the meantime. many thanks for your help.   73, Graeme ZL2APV   On Saturday, November 14, 2020 at 11:03:46 AM UTC+13 ahop...@googlemail.com wrote:

Back to top

You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to sparksdr+u...@googlegroups.com.

[ahop...@googlemail.com]

Hi Ward,

that looks very interesting. I had a very brief look at something I found on 'Federated Wiki' and admit I came away no wiser:) I look forward to seeing what this grows into.  The spot data will probably change going forward and I'm open to suggestions here, I want to add a means to tie spots to the receiver and radio that decoded it and I'm thinking of giving radios and virtual receivers friendly names/tags.

73 Alan M0NNB

[Ward Cunningham]

Oops. I see that by replying to a digest post that I have accidentally included the entire digest in my message. I'm sorry for that. I also see that the image I posted did not show through on the subsequent digest but I can see the image if I go to the groups.google.com site to read the post. Next I will try authoring in the website instead of my mail program. More soon.

[Ward Cunningham]

Alan -- Yes, I understand that this is an experiment and subject to change. Let me say I like the direction you are going and will have lots to play with first duplicating some things I did with the WSJT-X UDP packets which were much harder to parse but not impossible. I will summarize my experience eventually but first I will see what new I can do with what you have already.

My approach will be to write a variety of html scripts in single files that should work for anyone who has SparkSDR running on the same computer. I will make a list of things I'm doing on this page:

    http://ft8.ward.asia.wiki.org/spark-decodes.html

This morning I wrote a new script that plots the number of spots in every WebSocket message.

    http://ft8.ward.asia.wiki.org/assets/pages/spark-decodes/bands.html

Here is what I see when leaving this running from 8am to 1pm:

Best regards -- Ward

[Ward Cunningham]

I left my traffic counter running overnight and well into the following day (today) while I chased the activity around with my four radios. Here is the plot I had when I finally closed the window and went on to further development. I'm pretty sure this would be easier to look at and probably more meaningful if I summed adjacent even and odd time periods. This was worth doing once just to get an idea how "unbalanced" the two halves of the FT8 cycle become. And, yes, the peak decoded spots at one point today was 55 on 20 meters.

[J P Watters]

Ward,

I am trying to understand what these these pages tell me. ie. bands, signals, etc. 

I am running SparkSDR on a mac mini running catalina. I loaded MAMP so I can server web pages from it.

To escape the bondage to "localhost" I changed the websocket connect string to document.domain instead of localhost. 

ie const socket = new WebSocket('ws://'+ document.domain + ':4649/Spark')

Then I can run the websockets on another computer. I have not tried to figure out how to make the web pages more portable.

For example, I think using some of the bonjour code ("https://github.com/watson/bonjour") could make the pages portable and auto discoverable,

Then your wiki code could just discover SparkSDR running and collect data if you wanted. to. 

So much to learn.

..jpw J P Watters

KC9KKO

Morris, IL USA

[J P Watters]

Ward,

I left the band.html run and I attached an image. 

Can you tell me how you interpret this graph. It has 7.0/18.1/14/10 graphs.

..jpw J P Watters

KC9KKO

Morris, IL USA

[Ward Cunningham]

On Nov 17, 2020, at 11:11 AM, J P Watters <jpwa...@gmail.com> wrote:

Ward,

I left the band.html run and I attached an image. 

Can you tell me how you interpret this graph. It has 7.0/18.1/14/10 graphs.

..jpw J P Watters

KC9KKO

Morris, IL USA

J P — The chart shows FT8 activity on bands for which you have radios as measured by spot counts. This is the number that increments the counts shown for each radio in the toolbar display. Here, for example, I see 15680 spots decoded on 20 meters. In my graph I show the increment for each 15 second interval on the vertical axis. I’ve seen this run as high as 55. This is a very impressive number. 

I will have to add some logic that breaks the line for a given band when reception is suspended. Right now when you hop form band to band the plotting package I’m using connects the last spot count when you leave a band with the first spot count when you return.

At your suggestion I’ve added a parameter where one can specify the domain name or IP address for the computer running SparkSDR. I’ve specified domain=localhost to all the links on my “Spark Decodes” page. 

I’ve used the same parameter mechanism to add a variation to the script that watches individual stations come and go. This replaces the snr numbers with a rainbow color ranging from blue to red to yellow to green as signals get better.

I’ve had a lot of fun writing these scripts so kudos to Alan for thinking of this extension mechanism and putting up with me while I searched for the web socket enable option.

Best regards — Ward

[J P Watters]

Ward,

Cool, you are doing a really cool thing. 

Since I am running MAMP, I would think that storing the data in a SQL database would be a good idea.

With that queries could be run post reception to reveal other things. 

I have been looking for a way to rate a band for coverage, to various regions and determine the best time of day for contacts on that band and region of the world. 

I will have to go to github and update the code I have.

Now if I could only figure out how to best connect two HL2s to an antenna using only one of them to transmit with and FT8 contacts. Using T/R relays or circulators/combiners is my dilemma.

..jpw J P Watters

KC9KKO

Morris, IL 

[Ward Cunningham]

J P — I’m an on and off radio operator and a full time computer programmer. I enjoy hamming more if I am fiddling around with programs. I was active logging contacts on 20 meter FT8 for about a year. I was running my aging ICOM 735 into a $200 Windows 10 box that I bought specifically to run WSJT-X. I streamed data into various stores and then wrote programs that would visualize them within moving windows of from 10 minutes to a few days. All this was built on the UDP packets that WSJT-X would emit but I’m happy to leave most of that behind.

Here are my notes from 2-way operating that stopped abruptly this summer when a typo crashed my pc.

http://found.ward.bay.wiki.org/view/k9ox-ft8-stats/view/k9ox-ft8-maps

With that system out of the way I moved my antenna permanently to my HL2 which works so well I enjoy just thinking about it. When Alan added the web socket interface I’ve been all in on programming the things that were hard for me to do with the UDP payloads. Thank you Alan.

I would like to merge the decodes from my son’s radio and my own. I was doing that with the UDP setup. We’re 300 miles apart which meant we saw the bands differently. More interesting, we saw distant signals fade in sync but out of phase. Our guess is that’s do to the respective polarization of our antennas. I’d like to see that more clearly. Maybe we try some experiments like that together.

You mentioned MAMP. I wonder if you are programming that and what languages you’ve chosen? I’m pretty attracted to javascript but I’ve programmed most things in my career.

I”ve been rambling just waiting for my revised bands.html script to collect some data. It now averages adjacent time slots which was supposed to make the curves less jagged. With this first data it is hard to say if I have been successful. I also break the line if a band is quiet (or turned off) for over a minute. I can see the 5.5 MHz activity disappear for minutes while 3.5 MHz takes off. 

I’ll see what I have in the morning when the MUF starts climbing. 73 until then.

Best regards — Ward

On Nov 17, 2020, at 7:03 PM, J P Watters <jpwa...@gmail.com> wrote:

Ward,

[J P Watters]

Ward,

I used to program regularly, but now I tinker with code now and then.

MAMP it just a group of components to do some web development.

Apache, MySQL server. There is a windows version WAMP also.

https://documentation.mamp.info/

I loaded it simply so I could web serve the sparksdr web sockets code.

Then I could use an iPad or tablet over the wireless 

I have done some python, some jscript, but not really much. 

With the discovery of HL's I have been wandering on the nomadic journey to 

add them to my lonely TS2000. I have space with another chainsawing event to hoist a pair of 

Alpha Delta Parallel Dipole Antennas DX-LBPLUS antennas 90 degrees apart in an Vertical Inverter "V"

configuration. Until the chainsaw roars, I am pondering how to use the pair of HL2's that I have to run 

enough receivers to listen, skimming entire the band. The 10rx gateware does that, but no transmit. 

Running a second HL2 with standard 4rx gateware to use for the transmitter. I don't think that 

using a coupler with 30db isolation will quite protect the opposing transceiver from the transmit RF of the transmitting HL2.

I just haven't come up with a design.

Attached is the png of the bands that ran overnight.

[Ward Cunningham]

We have about half of one of our favorite displayed almost done. This listens for confirmed contacts and then includes them on a bubble and arrows diagram. Inward pointing arrows show a desirable station which we hear worked by many other stations. Outward pointing arrows means we’re hearing a “wolf” working one station after another.

Of course, many, much more complicated relationships show when you listen longer.

This script has lots more work to do such as various strategies for color coding and interactively rendering it with the Graphviz rendering program compiled into javascript. An early distribution joins other scripts online here: http://ft8.ward.asia.wiki.org/spark-decodes.html

[Ward Cunningham]

We now show confirmed receptions on a chosen band and update that display as new R’s arrive.

http://ft8.ward.asia.wiki.org/spark-decodes.html

On Nov 18, 2020, at 10:25 PM, Ward Cunningham <ward.cu...@gmail.com> wrote:

We have about half of one of our favorite displayed almost done. This listens for confirmed contacts and then includes them on a bubble and arrows diagram. Inward pointing arrows show a desirable station which we hear worked by many other stations. Outward pointing arrows means we’re hearing a “wolf” working one station after another.

<PastedGraphic-9.png><PastedGraphic-8.png>

Of course, many, much more complicated relationships show when you listen longer.

<PastedGraphic-10.png>

[Ward Cunningham]

New script unravels decodes into two-way exchanges that make complete contacts. We display these with each in its own cell shown in gray until we see the 73 conclusion. We look back for the CQ that starts the contact and has the Maidenhead coordinate for that station. A button will download all of the related spots for further analysis. 

http://ft8.ward.asia.wiki.org/spark-decodes.html

(See the web groups if your email doesn’t show the screenshot.)

[ahop...@googlemail.com]

Hi Ward,

this all looks fun.  A previous release of spark did put ft8/jt9 in a conversation view which I keep meaning to revive.

73 Alan M0NNB

[Ward Cunningham]

Alan — I was unaware of conversation view being fairly new to SparkSDR. Do you have pictures?

My current interest is simply discovering what hams actually do. My hope is that by developing technique in this domain I might have ideas for understanding what engineers do when resolving incidents. Thats my day job and intractable by comparison. 

CQ VE3XET EN58
VE3XET K5TCB EM45
VE3XET K5TCB EM45
VE3XET K5TCB EM45
VE3XET K5TCB EM45
K5TCB VE3XET +08
VE3XET K5TCB R-11
K5TCB VE3XET +08
VE3XET K5TCB R+03
K5TCB VE3XET +08
VE3XET K5TCB R+04
K5TCB VE3XET +08
VE3XET K5TCB R+00
K5TCB VE3XET RRR
VE3XET K5TCB 73
K5TCB VE3XET RRR
K5TCB VE3XET RRR
K5TCB VE3XET RRR
K5TCB VE3XET RRR
K5TCB VE3XET RRR
K5TCB VE3XET RRR
VE3XET K5TCB R+06
K5TCB VE3XET RRR
VE3XET K5TCB R+06
K5TCB VE3XET RRR
K5TCB VE3XET RRR
K5TCB VE3XET RRR

Here is a conversation I noticed this morning. I’m tempted to write a pattern matcher and use it to search for and classify these “unexpected” exchanges. Just counting each case and comparing the counts from band to band would be interesting. Perhaps I could visualize troublesome paths as a function of time of day. If more interesting questions occur to you, please share.

I wrote his retrospective on yesterday’s hacking.

http://ft8.ward.asia.wiki.org/making-the-exchange-script.html

Best regards — Ward

-- 
You received this message because you are subscribed to a topic in the Google Groups "SparkSDR" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sparksdr/wN4tkhi08yg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sparksdr+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/4b7d753a-0bb9-499c-8636-27194e3e476cn%40googlegroups.com.

[ahop...@googlemail.com]

Hi Ward, the screen shots here http://www.ihopper.org/radio/ have the old view, really need to update that page.

[Ward Cunningham]

Alan — You have inspired me to offset the odd time slots which helps read exchanges with similar calls. Thanks — Ward

CQ AA1ON FN42
  AA1ON AJ6MV CM87
AJ6MV AA1ON +03
  AA1ON AJ6MV R-04
AJ6MV AA1ON RR73
  AA1ON AJ6MV 73

[Ward Cunningham]

The offset he’s even more when the response takes a few tries.

CQ KE6I CM87
  KE6I K2BAG DM33
  KE6I K2BAG DM33
  KE6I K2BAG DM33
  KE6I K2BAG DM33
K2BAG KE6I +05
  KE6I K2BAG R-03
K2BAG KE6I RR73
  KE6I K2BAG 73

[Ward Cunningham]

I’ve added click to highlight a string everywhere. Here I watch one CQ starts two contacts, nicely interleaved.

(See web groups for image)

Without clicking it searches for K1JT KF7O or M0NNB. We’re watching for you.

Best regards — Ward