"Top Monitors by reports over last 24 hours" Calculation Questions
160 views
Skip to first unread message
Steve Haynal
Jul 10, 2020, 2:25:00 AM7/10/20
to PSK Reporter
Hi Group,
I am trying to understand exactly how the top monitors by reports over last 24 hours are calculated. I know there is some filtering, but I must not be accounting for all the filtering because I see discrepancies when examining my .adif logs from pskreporter. Here is the experiment I ran and which anyone else can run too:
First, I decided to look at data for a relatively inactive FT8 band, 12M. Next, I looked at my standing in the top monitors list for 12M and I had 399 reports. I immediately downloaded my .adif log for the past 24 hours. I did these last two steps late at night when I know I haven't made more than a handful of 12M spots in the last few hours so that the server had time to "settle" the data and ensure the reported spots agreed with the logs. Next, I applied the attached python script to count unique spots. The script counts one unique spot for each call reported at least once in a 20 minute window for 12M. My script says there are 530 unique spots. The data fed to the python script is from pskreporter so I know pskreporter has a record of all 530 spots. I would like to understand what additional filters pskreporter is applying to reduce the reported spot count to 399. I have attached my python script and my .adif log for 12M spots in 24 hours. It should be possible to repeat this experiment with similar results at any time by grabbing new numbers and logs from pskreporter for kf7o on 12M.
The python script uses this adif module:
73,
Steve
kf7o
Steve Haynal
Jul 10, 2020, 2:46:22 AM7/10/20
to PSK Reporter
Hi Group,
When I run the same experiment for WZ7I, the leader board reports 406 spots and my script reports 407 spots for 12M from his .adif log. This gives me confidence in my script. It also makes me want to get to the bottom of why there is a discrepancy. WZ7I is using a Red Pitaya. I am using a Hermes-Lite 2.0 with SparkSDR.
73,
Steve
kf7o
Philip
Jul 10, 2020, 9:00:11 AM7/10/20
to PSK Reporter
I've stared at the code, and I think that you should have been credit for 445 (or thereabouts) rather than the 399. The only explanation that I can think of is that you are reporting spots (sometimes) which are more than 2 hours old. The stats generation process maintains an intermediate table and the last 2 hours of it get recomputed twice every hour. If I do a manual run over the last 24 hours, then it doesn't agree (for a couple of 20 minute periods). The only way that I can think of this happening is that some spots got delayed for more than 2 hours.
I will also note that I see a bunch of bad callsigns being reported:
2020-07-10 11:55:01.380211500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'DX'
2020-07-10 11:55:01.380223500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'ARIGAT'
2020-07-10 11:55:01.384661500 kf7o (SparkSDR v2.0.1.9) FT8: Inserting new TX station 'K2TAW' in 'EK80' (from 'EK80')
2020-07-10 12:00:01.378121500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'ASIA'
2020-07-10 12:00:01.385486500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'ANT'
2020-07-10 12:00:01.391739500 kf7o (SparkSDR v2.0.1.9) FT8: Inserting new TX station '0000OAPGXFZ' in '' (from '')
2020-07-10 12:05:01.386338500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'AS'
2020-07-10 12:10:01.382118500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'NA'
2020-07-10 12:10:01.382671500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'DX'
2020-07-10 12:10:01.382825500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'DX'
2020-07-10 12:10:01.384877500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'AS'
2020-07-10 12:15:01.383974500 kf7o (SparkSDR v2.0.1.9) FT8: Dropping reception record -- invalid sender callsign 'DKJ'
I'm also a bit surprised that I didn't think that 0000OAPGXFZ was invalid!
Philip
Alan Hopper
Jul 10, 2020, 9:24:51 AM7/10/20
to PSK Reporter
Hi Steve and Philip,
I'll add some metrics in to check for delays in sending, I'd be very surprised if can get to 2hrs but maybe on an overloaded pc. I'll improve my callsign checker which is obviously wanting, If anyone has a good set of rules I'd be very interested.
73 Alan M0NNB
Philip
Jul 10, 2020, 10:53:28 AM7/10/20
to PSK Reporter
I *thought* that the rule that I used was that a valid callsign had to match (essentially)
[0-9]*[a-z]+[0-9]+[a-z]+
However, it appears to be much more lax (and there are complications with '/' and '-' characters). However, This is a list of recently spotted callsigns, and I suspect that they are all bogus. If anyone can see a good callsign, please point it out:
+-------------+
| callsign |
+-------------+
| 390ZCZ |
| GP18 |
| 161MSU |
| 55VYV |
| 498UVR |
| 569TAP |
| 412ZX |
| 252WRO |
| 15DVX |
| 417ZUY |
| 933DD |
| 655FPF |
| 926QUN |
| 99HJK |
| 233TFX |
| 358ZCZ |
| 34LFR |
| 326XXC |
| 310SDS |
| 811FET |
| 099QEP |
| 514XAL |
| 033XGY |
| 836KXI |
| 316CLW |
| 811VAN |
| 583NKC |
| 16QEC |
| 1TOAEAZQ9 |
| 193SZL |
| 15ZJL |
| 634VGB |
| 644ENE |
| 592QIF |
| 555IQB |
| 679MWG |
| 995YGI |
| 17PWX |
| 3OGSQEABM |
| 001JJO |
| FB21 |
| KP86 |
| 138FZH |
| 644LH |
| 910KLK |
| 762OUU |
| 174CMH |
| 423FPC |
| 349LML |
| 0000OAPGXFZ |
| 393NSL |
| 95ZWV |
| 512ABK |
| 00278PU91 |
| QT23 |
| KRV7 |
| 122VFW |
| 8NNI |
| 405IBQ |
| 639TS |
| 941OHA |
| 47RTQ |
| 771FJM |
| 208BAZ |
| JUIZUX4 |
| BHQUOKSR9 |
| 270QZD |
| 155WKQ |
| 907XEO |
| 5ZOWHCZJ |
| 808JVD |
| 0YKL |
| 518NNO |
| 612YPP |
| 510GTE |
| 335UMD |
| 6QZNKRLWP |
| 782IYC |
| 00000000S |
| 5ICT |
| 7TGJOPPI596 |
| 397EOX |
| 673VDI |
| 15I4- |
| 972P |
| 222FLU |
| 949HXS |
| 752IAP |
| VKMI7 |
| PTBJ6 |
| 393XBI |
| 684GBN |
| 228AVT |
| 023PQH |
| 646PFF |
| 73UPM |
| 2UWCENXM8 |
| 63IVB |
| 066HRT |
| KARIM73 |
+-------------+
Steve Haynal
Jul 10, 2020, 1:13:50 PM7/10/20
to PSK Reporter
Hi Philip and Alan,
Thanks for taking a look. I assume the 445 reports is from a more recent grab of my 12M .adif log? I added a regular expression call sign check to spots_v2.py, attached, and reran it on my .adif log that I attached last night. I still see 530 valid-looking spots. I did a visual inspection of the calls and didn't see anything bogus. I also downloaded my most recent complete .adif log and didn't see any of the "invalid sender callsigns" of "NA" "AS" "DX" you reported either with my script or with simple grep. So I assume your filter works for these and blocks them from the logs and the leader board. I also never found 0000OAPGXFZ in any of the .adif data I have which I think spans the time you were looking.
I'll investigate the possibility of late reports via the .adif logs. I will grab my log every hour today, and then see if spots more than 2 hours old are inserted when comparing two logs. Each new log should only have new spots for the last hour.
73,
Steve
kf7o
Steve Haynal
Jul 10, 2020, 1:27:10 PM7/10/20
to PSK Reporter
Hi Philip,
I believe these are recent bogus spots for all your reporters, not just me?
I applied the rule [0-9]*[a-z]+[0-9]+[a-z]+ to my 24 hour all band .adif log and it flags the following as bogus:
Bad Call 15m
Bad Call w2/eu1ki
Bad Call ft4
Bad Call em54
Bad Call en82
Bad Call rr73
Bad Call k-0690
Bad Call w2/s50sd
Bad Call k-0690
Bad Call w2/s50sd
Bad Call w2/s50sd
Bad Call kb2
Bad Call 40m
Bad Call b1/bh3qby
Bad Call kh0/kc0w
Bad Call b1/bh3qby
Bad Call b1/bh3qby
Bad Call b1/bh3qby
Bad Call b1/bh3qby
Bad Call b1/bh3qby
Bad Call kh0/kc0w
Bad Call b1/bh3qby
Bad Call du6/n6ss
Bad Call as047
Bad Call bw2/ja1qqv
Bad Call bw2/ja1qqv
Bad Call ft4
Bad Call ft4
Unique spots 18709
Some of these still look valid. If I switch from match to search (find just a valid callsign within the call), then the results look better to me:
Time buckets 72
Bad Call 15m
Bad Call ft4
Bad Call em54
Bad Call en82
Bad Call rr73
Bad Call k-0690
Bad Call k-0690
Bad Call kb2
Bad Call 40m
Bad Call as047
Bad Call ft4
Bad Call ft4
Unique spots 18725
Note that this is just 12 bogus calls out of almost 19000 reports across all bands,
73,
Steve
kf7o
Philip
Jul 10, 2020, 2:04:42 PM7/10/20
to PSK Reporter
Yes -- the list of 100 that I sent was for all reporters. The original list higher up was just for you -- and was mostly two character strings like 'DX'.
These are the counts (after recalculation) that I have for you on 12m for each 20 minute period -- how do these line up with what you think?
+---------------------+-------+
| from_unixtime(per) | rseen |
+---------------------+-------+
| 2020-07-09 13:20:00 | 1 |
| 2020-07-09 14:00:00 | 4 |
| 2020-07-09 14:20:00 | 8 |
| 2020-07-09 14:40:00 | 20 |
| 2020-07-09 15:00:00 | 14 |
| 2020-07-09 15:20:00 | 22 |
| 2020-07-09 15:40:00 | 30 |
| 2020-07-09 16:00:00 | 27 |
| 2020-07-09 16:20:00 | 39 |
| 2020-07-09 16:40:00 | 34 |
| 2020-07-09 17:00:00 | 31 |
| 2020-07-09 17:20:00 | 26 |
| 2020-07-09 17:40:00 | 5 |
| 2020-07-09 18:40:00 | 23 |
| 2020-07-09 19:00:00 | 27 |
| 2020-07-09 19:20:00 | 26 |
| 2020-07-09 19:40:00 | 18 |
| 2020-07-09 20:00:00 | 27 |
| 2020-07-09 20:20:00 | 9 |
| 2020-07-09 20:40:00 | 2 |
| 2020-07-09 21:20:00 | 3 |
| 2020-07-09 21:40:00 | 11 |
| 2020-07-09 22:00:00 | 7 |
| 2020-07-09 22:20:00 | 5 |
| 2020-07-09 22:40:00 | 7 |
| 2020-07-09 23:00:00 | 8 |
| 2020-07-09 23:20:00 | 2 |
| 2020-07-09 23:40:00 | 6 |
| 2020-07-10 00:40:00 | 1 |
| 2020-07-10 01:20:00 | 1 |
| 2020-07-10 02:20:00 | 1 |
| 2020-07-10 13:40:00 | 1 |
| 2020-07-10 14:00:00 | 9 |
| 2020-07-10 14:20:00 | 14 |
| 2020-07-10 14:40:00 | 13 |
| 2020-07-10 15:00:00 | 10 |
| 2020-07-10 15:20:00 | 23 |
| 2020-07-10 15:40:00 | 20 |
| 2020-07-10 16:00:00 | 28 |
| 2020-07-10 16:20:00 | 23 |
| 2020-07-10 16:40:00 | 19 |
| 2020-07-10 17:00:00 | 25 |
| 2020-07-10 17:20:00 | 17 |
| 2020-07-10 17:40:00 | 6 |
+---------------------+-------+
Steve Haynal
Jul 10, 2020, 3:07:29 PM7/10/20
to PSK Reporter
Hi Philip,
Here is what I glean from the .adif logs from pskreporter. I'm actually not logging locally, just via your site. Most buckets match, but those that don't are marked with *. Buckets I have but your don't are marked with **. I did not see any duplicate or bogus calls in each bucket.
*20200709_1740 24
**20200709_1800 27
**20200709_1820 26
*20200709_1840 25
20200709_1900 27
20200709_1920 26
20200709_1940 18
20200709_2000 27
20200709_2020 9
20200709_2040 2
20200709_2120 3
20200709_2140 11
20200709_2200 7
20200709_2220 5
20200709_2240 7
20200709_2300 8
20200709_2320 2
20200709_2340 6
20200710_0040 1
20200710_0120 1
20200710_0220 1
20200710_1340 1
20200710_1400 9
20200710_1420 14
20200710_1440 13
20200710_1500 10
20200710_1520 23
20200710_1540 20
20200710_1600 28
20200710_1620 23
20200710_1640 19
20200710_1700 25
20200710_1720 17
*20200710_1740 10
Total unique spots 475
I think the last bucket is different because I grabbed the logs at a slightly later time. At the very beginning of the 24 hour period there appear to be more spots according to the .adif logs than in your list. Below are the unique calls I have for those buckets. In your list there was no activity reported for the 20200709_1800 bucket but according to the .adif logs it was a very active 20 minutes. I scrolled back in the SparkSDR logs and there was indeed activity during that 20 minute period. The calls in the SparkSDR logs agree with the ones printed below from the .adif logs.
Do you think it is these spots which are somehow added more than 2 hours after the real spot time which are throwing off my leader board numbers? I am collecting logs every 1 hour today and will be able to see if there are spots more than 2 hours old being added.
20200709_1740 24
wb2vuo wa6poz aa5be w4dsm kc9eci k9rx w0mdk nv9l ve3yxo n1ix ac2pb k1jd wb9ami nk5b k4ag n6gr w7zr n1mgo k4izn ai2n ks0aa n6tto w7osg co8ly
20200709_1800 27
w5gj xe1kk wb0mco n0igu wa7cpa ab4wl kd8jsp k9kds n2jnr kc8pv n5xz nr5t k9nr k4em n7kdt k9exy kg5hth ka6rwl wa0csl ve3yxo w4pot n6gr k9zo w7zr kt5p n5jxo n5ll
20200709_1820 26
ve3qk n0igu aa4mj k2ak kg5zmq k5ot k9kds wi9d w7ysb w0mdk nr5t k4nkt n7kdt wa8ick k0mld w5etj w5afv ke5az wb9ami n6gr w7zr n7mdf k5rz k4izn ke0dc co8ly
20200709_1840 25
w5gj wa7cpa n0igu ka8g ab7rw w0pe n5ob k5ppp n8lj n7kdt k4mq k7cdz kd4js k1jd w5afv wb5bhs wi7n w4mps w7zr k0jv ka9war tg9akh w9dx ke0dc n5wa
73,
Steve
kf7o
Philip
Jul 10, 2020, 3:48:12 PM7/10/20
to PSK Reporter
Ah -- it turns out that you are being caught by the bad QRG filter. Your callsign was marked as misreporting on (I guess) 12m for 45 minutes.
2020-07-09 17:51:41.372531500 KF7O,12m now appears to be bad: 0 1
Based on what you are doing, I expect that this is a false positive. I need to dig into what actually happened and see if I can adjust the code slightly.
The original problem is that a bunch of reporters report the wrong frequency -- probably because their CAT connections are not set up correctly. The system assumes that in a particular 15 second interval, a callsign only transmits on a single band, and it tries to figure out the "correct" band based on how trustworthy the previous reports have been.
Steve Haynal
Jul 10, 2020, 4:28:07 PM7/10/20
to PSK Reporter
Hi Philip,
Thanks for investigating this. Please let Alan and me know if there is anything we can or should do differently to not be flagged by the QRG filter. I've used WSJT-X and made the mistake of reporting spots to the wrong band in the past. But SparkSDR is highly integrated and I don't even know if/how it could be setup to report spots for the wrong band. Maybe it is unavoidable, but it would be unfair to flag SparkSDR for QRG based on two WSJT-X or other reporters reporting on the wrong band. Maybe some weight can be given to the reporting software based on how easy or hard it is to make this mistake. For instance, it would be very hard to make this band mistake with Red Pitaya or SparkSDR, but easy with WSJT-X.
73,
Steve
kf7o
Philip
Jul 10, 2020, 4:46:44 PM7/10/20
to PSK Reporter
What timestamp do you use for the connection? I think that WSJT-X uses the decode time (which is *after* the complete transmission has been received). I.e. for the first message of the minute, this probably means that the timestamp would be between 13 and (maybe) 18 seconds past the minute.
The particular case that bit you was a transmitter (XE1KK) who kept on switching bands between 12m and 6m. I wonder if this is a timestamp problem for you....
Steve Haynal
Jul 10, 2020, 4:55:46 PM7/10/20
to PSK Reporter
Hi Philip,
This makes sense. If SparkSDR is reporting spots 12-18 seconds "later" than everyone else, then SparkSDR may be flagged for QRG as it looks like we are reporting a call on a band when everyone else reports the same call on another band. Alan, the author of SparkSDR, uses WSJT-X for decoding under the hood and can share exactly what time he uses for the TIME_ON field. It should be easy enough to subtract 15 seconds in SparkSDR if this is the issue.
I've noticed in the logs that sometimes the FT8 times are reported 1 second before the 15 second mark, :14 or :29, in SparkSDR. How close to the 15 second mark should a spot time be to not cause problems for pskreporter?
73,
Steve
kf7o
Philip
Jul 10, 2020, 5:27:55 PM7/10/20
to PSK Reporter
From looking at the code, it uses a 10 second offset -- i.e. all reported times in the range xx:yy:10 - xx:yy:25 will be considered to be in the same time period.
Philip
Steve Haynal
Jul 11, 2020, 1:00:26 AM7/11/20
to PSK Reporter
Hi Philip,
I did a little more sleuthing in the .adif logs and when comparing my spots with SparkSDR to spots of the same call/band by others, I see strong evidence that SparkSDR is off by 15 seconds as you say. Just to make sure I understand, if a FT-8 spot is received from 18:00:00 to 18:00:14, the "TIME_ON" field or flowStartSeconds should be set to 18:00:00 or the equivalent unix time?
Also, I finished my comparison of logs every hour for my logged spots on 12M. The columns below are the 13 times (PDT) that I downloaded a log. The rows are the 72 20-minute buckets in a day. I've dropped the first and last 20-minute bucket from each column. As you can see, spots never increase in the log for previous buckets. This rules out the possibility that SparkSDR was reporting spots more than 2 hours late.
I think Alan and I now know what to do. First, align SparkSDR spot times with the rest of the world so that SparkSDR is not penalized for QRG. Second, do not send spots with bogus or questionable call signs.
Thanks for you help and 73!
Steve
kf7o
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| Bucket | 0951 | 1051 | 1151 | 1251 | 1351 | 1451 | 1551 | 1651 | 1751 | 1851 | 1951 | 2051 | 2151 |
+===============+========+========+========+========+========+========+========+========+========+========+========+========+========+
| 20200709_1640 | | | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1700 | 31 | | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1720 | 26 | | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1740 | 35 | | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1800 | 27 | 27 | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1820 | 26 | 26 | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1840 | 25 | 25 | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1900 | 27 | 27 | 27 | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1920 | 26 | 26 | 26 | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_1940 | 18 | 18 | 18 | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2000 | 27 | 27 | 27 | 27 | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2020 | 9 | 9 | 9 | 9 | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2040 | 2 | 2 | 2 | 2 | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2120 | 3 | 3 | 3 | 3 | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2140 | 11 | 11 | 11 | 11 | 11 | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2200 | 7 | 7 | 7 | 7 | 7 | 7 | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2220 | 5 | 5 | 5 | 5 | 5 | 5 | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2240 | 7 | 7 | 7 | 7 | 7 | 7 | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2300 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2320 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200709_2340 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_0040 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_0120 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_0220 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1340 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1400 | 9 | 9 | 9 | 9 | 9 | 9 | 9 | 9 | 9 | 9 | 9 | 9 | 9 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1420 | 14 | 14 | 14 | 14 | 14 | 14 | 14 | 14 | 14 | 14 | 14 | 14 | 14 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1440 | 13 | 13 | 13 | 13 | 13 | 13 | 13 | 13 | 13 | 13 | 13 | 13 | 13 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1500 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1520 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1540 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1600 | 28 | 28 | 28 | 28 | 28 | 28 | 28 | 28 | 28 | 28 | 28 | 28 | 28 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1620 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1640 | | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1700 | | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1720 | | 17 | 17 | 17 | 17 | 17 | 17 | 17 | 17 | 17 | 17 | 17 | 17 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1740 | | | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 | 23 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1800 | | | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1820 | | | 21 | 21 | 21 | 21 | 21 | 21 | 21 | 21 | 21 | 21 | 21 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1840 | | | | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1900 | | | | 26 | 26 | 26 | 26 | 26 | 26 | 26 | 26 | 26 | 26 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1920 | | | | 33 | 33 | 33 | 33 | 33 | 33 | 33 | 33 | 33 | 33 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_1940 | | | | | 33 | 33 | 33 | 33 | 33 | 33 | 33 | 33 | 33 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2000 | | | | | 29 | 29 | 29 | 29 | 29 | 29 | 29 | 29 | 29 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2020 | | | | | 29 | 29 | 29 | 29 | 29 | 29 | 29 | 29 | 29 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2040 | | | | | | 17 | 17 | 17 | 17 | 17 | 17 | 17 | 17 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2100 | | | | | | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2120 | | | | | | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2140 | | | | | | | 16 | 16 | 16 | 16 | 16 | 16 | 16 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2200 | | | | | | | 18 | 18 | 18 | 18 | 18 | 18 | 18 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2220 | | | | | | | 18 | 18 | 18 | 18 | 18 | 18 | 18 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2240 | | | | | | | | 17 | 17 | 17 | 17 | 17 | 17 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2300 | | | | | | | | 13 | 13 | 13 | 13 | 13 | 13 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2320 | | | | | | | | 16 | 16 | 16 | 16 | 16 | 16 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200710_2340 | | | | | | | | | 20 | 20 | 20 | 20 | 20 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200711_0000 | | | | | | | | | 9 | 9 | 9 | 9 | 9 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200711_0020 | | | | | | | | | 4 | 4 | 4 | 4 | 4 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200711_0040 | | | | | | | | | | 5 | 5 | 5 | 5 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200711_0100 | | | | | | | | | | 1 | 1 | 1 | 1 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200711_0120 | | | | | | | | | | | | 1 | 1 |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| 20200711_0300 | | | | | | | | | | | | | |
+---------------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
Alan Hopper
Jul 11, 2020, 1:43:23 AM7/11/20
to PSK Reporter
Hi Steve and Philip,
Thanks very much for this, It should be easy to fix. Looking at my code I'm using the start time of the reception for flowStartSeconds so I'm probably 15s earlier than wsjtx, I'll do some tests and make sure it is the same as wsjtx as I guess that is the defacto standard.
73 Alan M0NNB
Alan Hopper
Jul 11, 2020, 4:22:36 AM7/11/20
to PSK Reporter
Hi Steve and Phillip,
I've been looking at my code and WSJTX, I am using the reception start time which is the same as the spot time seen on the wsjtx and spark displays and I have checked that they all agree. WSJT-x appears to use the time that the decode finishes which can be sometime after the end of the 15s slot and is a bit variable. Spark queues up decodes when things get busy so I can't use that time, I think the best I can do is add 15s or so to the time I use but this feels a bit of a hack.
73 Alan M0NNB
Philip
Jul 11, 2020, 8:23:03 AM7/11/20
to PSK Reporter
It looks to me as though WSJT-X uses the time of the decode and you are using a time that is almost always align on the 15 second mark. The analysis below is, for sparksdr and wsjt-x, the recent ft8 spots, the time mod 15. You can see that wsjt-x reports between 11 seconds and 17 seconds -- I'm assuming that this is after the start of the 15 second slot. SparkSDR reports aligned on the 15 second boundary. I group the reports that are between 10 and 25 seconds past the start of the interval -- i.e. it covers the observed range of wsjt-x.
+----------------------------------+---------------------------+----------+
| substring(decoderSoftware, 1, 4) | mod(flowStartSeconds, 15) | count(*) |
+----------------------------------+---------------------------+----------+
| Spar | 0 | 8528 |
| Spar | 14 | 1048 |
| WSJT | 0 | 80706 |
| WSJT | 1 | 20190 |
| WSJT | 2 | 7373 |
| WSJT | 3 | 3363 |
| WSJT | 4 | 1648 |
| WSJT | 5 | 845 |
| WSJT | 6 | 469 |
| WSJT | 7 | 283 |
| WSJT | 8 | 178 |
| WSJT | 9 | 178 |
| WSJT | 10 | 425 |
| WSJT | 11 | 119680 |
| WSJT | 12 | 172381 |
| WSJT | 13 | 15528 |
| WSJT | 14 | 165980 |
+----------------------------------+---------------------------+----------+
Alan Hopper
Jul 11, 2020, 11:29:12 AM7/11/20
to PSK Reporter
Philip,
that's very useful and fits with what I believe both bits of software do. Adding 15s should put me in the right place and I think you have uncovered a minor bug in my code causing the ones at 14s.
Thanks Alan
Steve Haynal
Aug 4, 2020, 1:36:34 AM8/4/20
to PSK Reporter
Hi Philip,
I just wanted to follow up on this thread. Alan made the changes to the reporting time in SparkSDR and now everything is much more as expected. I'm often in the lead for my region of North America west coast! Search for "HERMES-LITE" in the top reporter rankings. It is hard for those of us on the US west coast to compete with Europeans or even those on the US east coast The count on pskreporter now agrees with my Python script analysis of 24 hour .adif logs to within 1%. This is much better than the ~30% difference I was seeing before. Thanks for all your help and work!
73,
Steve
kf7o
gordon wyatt
Aug 9, 2020, 12:20:18 PM8/9/20
to PSK Reporter
Hi all
I am no computer man being an old retired CRT and glass tubes man.
I have been on 4mtrs all day and been heard or worked in 14 countries, However I have a few mins ago worked 9H1CG and been heard by OY1OF my total after waiting a long time, is still stuck on that 14 DXCC rather than 16
Wondering if the above two Islands are not included in the DXCC entities along with GJ,GU,GD.
Looking at the full 24 hours all those from last evening have gone but they were just G's anyway.
73 Gordon GW8ASA
Philip
Aug 9, 2020, 1:37:23 PM8/9/20
to PSK Reporter
The scores on the statistics page are recomputed once per hour -- so they can be up to an hour out of date....
gw8...@gmail.com
Aug 9, 2020, 3:40:39 PM8/9/20
to PSK Reporter
Thanks Philip
Still stuck on 14 on my return from F1 race :-)
Could it be a mix of modes between FT8 and MSK144
It's not important.
73 Gordon 8ASA
Tsepkovsky Sergey
Sep 30, 2020, 1:52:19 AM9/30/20
to psk-re...@googlegroups.com
--
You received this message because you are subscribed to the Google Groups "PSK Reporter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psk-reporter...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/psk-reporter/3a5b1132-cb27-4063-a0cc-f97699980b8dn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages