WSPR dupes when using 2 physical radios

[PE3ES / F4VTQ]

Hi Alan,

When doing WSPR with Hermes and RSP1a on the same bands I do see MANY duplicates in the WSPR uploads. Over 60% which for me means several thousands per day. Is SparkSDR not de-duplicating the decodes?

The algorithm to do that could be easily borrowed from for instance Red Pitaya Notes.

and consists of 3 commands sort - awk - sort done on the ALL_WSPR.TXT file for the same frequency. Processing time would be a few seconds per band.

I assume the same holds for FT8 or other digital modes.

On wspr.rocks there is even a top 200 / wall of shame for spotters doing too many dupes, ...

Did not check what software would be for the top positions yet ;-)

Bye

Erwin

[ahop...@googlemail.com]

Hi Erwin,

there is no deduping on uploads to wsprnet, there is deduping on all uploads to pskreporter.  No one has mentioned this before so you maybe the first using two radios on the same band on wspr. I shall add this to the wsprnet upload.

thanks for the report.

73 Alan M0NNB

[PE3ES / F4VTQ]

Thanks Alan,

I have looked a bit deeper into it as well.

It is not only the 2 radios that are creating dupes. The rate of dupes was much too high for the overlapping bands that I had set.

The main reason is wsprd, which needs a clean up process after getting the decodes done especially on deeper levels of decoding.

I looked at the statistics on wspr.rocks and analyzed the underlying cause.

An example or 2

Mostly they are of the form at the left, sometimes they might be like the one on the right. So the most numerous ones are within +/- 10Hz from the main signal. When using wspr for decoding it needs a second step to get rid of these deep decoding artifacts. As far as I know they are the results of doing a first decode and then subtract the signal of the first found peaks from the original signal and do a second decode run etc. Most software like WSJTx, JTDX, Pavel Demin Red Pitaya sw,  wsprdaemon run a process to sort by SNR and keep only the best of these as single callsign/frequency spots before uploading. The current dupe-rate for SparkSDR is over 60%.

My results with SparkSDR

                            totals        LF     MF   160m        80m        60m        40m        30m        20m        17m        15m        12m        10m       
raw spots        18,424        0        0        127        3,955        323        7,584        4,628        1,200        250        335        12        10       
uniques                 700        0        0        8        96        13        245        164        84        37        45        5        3        
duplicates       11,850        0        0        0        3000        166        4930        3748        6        0        0        0        0      
% dups        64.32%        Anything above a few % usually means the station is deliberately uploading dups from multiple receivers on the same band.
% unique        3.80%

Op maandag 28 februari 2022 om 09:32:44 UTC+1 schreef ahop...@googlemail.com:

[ahop...@googlemail.com]

Hi,

yep there is no de duping on wsprnet uploads so it will be the same for virtual receivers on one radio or multiple radios. I shall add it to the next version soon but it won't make the version I intend to release today as it would have meant another round of testing.

73 Alan M0NNB

[Mike Brown]

Hi Alan/Erwin

For information -  a report on dupes I'm seeing - my rate overnight was about 0.1% but may be 0%.

Since late last night I ran WSPR using my HL2 on 160m/80m/40m/30m and my RSP1A on 2m. All with SparkSDR 2.0.14.0 on a 9 year old i3 laptop running W10 Home 21H2 19044.1526. I can see no setting for WSPR decode depth so assume it is the same as Erwin's. I have the default SparkSDR setting of 1 core reserved. Could the limited processing power be reducing the decode depth?

Total spots when I looked at wspr.rocks a little while ago were just over 3,000. There were 3 duplicates (note that the wspr.rocks 'most duplicates' table counts both instances, so it reported me having 6 duplicates).

The 3 spots were of two stations, each received around 30 times in all. Logically you'd think that meant no TX problems. But. Those two stations were themselves at the number 2 and 4 positions for most received dupes. Both seem to be using versions of WSJT-X. This appears to suggest they may have significant problems, perhaps audio intermod between the radios and WSJT-X.

I have just stopped monitoring and will restart later using my 3 year old W11 i5 desktop to see if I can find more dupes.

Best 73

Mike

G4RAA

--
You received this message because you are subscribed to the Google Groups "SparkSDR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sparksdr+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sparksdr/18a1639e-e00a-431b-a85b-2836ea4013adn%40googlegroups.com.

[Mike Brown]

Ignore my previous email. Just realised the WSPR depth settings aren't with the others in the FT/JT/WSPR menu but the unnamed box next to the PSK Reporter button.

73

Mike

[ahop...@googlemail.com]

Hi All,

the 2.0.18 preview release tries not to upload any duplicate spots to wsprnet, this works across all decoders running in a single instance of spark.  Generally the only reason to run multiple instances on one pc is to use a different callsign for reporting, in this case duplicates between instances should not matter.  For the case of multiple computers the current solution will not de duplicate but this should only be an issue if you cover the same bands and modes on different computers.

73 Alan M0NNB