Testing guidelines

[Alex VE3NEA]

Dear beta testers of Morse Expert,

Thank you very much for volunteering to test my new app. I started this testing to discover and fix defects before the official release, and to do this, I need your input. Closed testing will last for about a week, so please provide your feedback soon.

Since some of you have no experience with testing, here are a few ideas what to test.

- Documentation: read the doc page available here and see if anything is not clear, if some information is missing, note any spelling errors.

- Installation: install the app and watch the installation process, note any unusual things that may happen: warnings and error messages, app freezing, etc. If you have the app installed, you can uninstall it and install again.

- Appearance: check if the app UI looks good, there are no elements that are too small or too large, no unusual colors.

- Commands: try all menu commands and buttons, see if they work as described in the doc.

- Save Text: see if you can save decoded text to a file and copy the file to your PC.

- Decoding: play back some CW audio from your radio, your keyer, or from YouTube. See if Morse Expert decodes the messages.

- Crashes: if the app freezes or crashes, please document as much detail as possible.

Please post your findings in this group. If you do not discover any defects, please let me know as well.

73 Alex VE3NEA

[ehrussell]

Hi Alex,

I'm using direct audio from my Flex 6700 to an iMic USB interface on my Note 8. So far there haven't been any installation, appearance, or command problems. And no crashes so far.

The UI seems fine. The waterfall seems a little skinny. Even so I'm able to select signals with my finger.

I didn't see in the help what the frequency bandpass is. It seems to be about 300-1000hz. The arrow tracks ok in that range but there is a small area at the top and bottom where I can see the signals in the waterfall but the arrow won't attach and decoding doesn't happen. Some explanation of this might be helpful. Also an option to do 3kz might be useful.

I'm surprised how slow most ops are on the band, and sloppy too. This gives the decoder some problems. It seems to be better at high speeds. Also it's important to keep the AGC under control. Still I don't think I'm seeing quite the quality of decoding that usually happens in CWS.

For a further test I'm setting up CWS in 3khz mode so I can directly compare decoding. Hopefully I'll have that result tomorrow.

Nifty little app!

73,

Ed / W2RF

[Phil Deaver]

Alex-

I'm afraid I didn't have much luck with the Morse Expert app on the device I tried it on. I have a LG Tablet.(LG G pad 8". running android 7.0, kernel 3.10.84) When I downloaded the app it had a message something like "this app is not optimized for this device" or something similar. I kept going and the app loaded. I held it close to the radio's speaker and even though the morse was pretty clear to me from the speakers, the app had alot of noise on it. It didn't always decode anything, even though you could clearly make out and read the CW pattern in the upper display. I also tried a second set of speakers. Occasionally it would decode OK but then just quit and not decode again. I'd say I had less than 25% decodes when the CW was pretty clear. I had CWget running on the PC as well and it decoded maybe 90%+ of the characters. Of course it had a better signal going into it. I tried running a cable from the phones output of my radio to the input of the LG tablet (without attenuating it) and it didn't work any better.

Let me know if there is something else I can try.

Phil WR7T

[KT9X - Lahra Flip Svare]

I used mine the same way I use Morse Decoder (google play store). Just holding my phone up to the speaker. But the Morse Expert barely decoded anything, unfortunately. Morse Decoder decoded about 80% of the same signal. If this app is intended to be used with audio cable input, I did not try that, as I have a computer for that. I tried it in place of Morse Decoder, but it did not decode as I hoped. Less than 5%.

Best of luck!

Flip

--
You received this message because you are subscribed to the Google Groups "Morse Expert for Android" group.
To unsubscribe from this group and stop receiving emails from it, send an email to morse-expert-for-a...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/morse-expert-for-android/fdd2cbbe-5c14-4d46-98af-7027f705dbf0n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex VE3NEA]

Thank you for your tests, guys! If the program is not decoding as expected, the first thing to check is which frequency it is decoding. Does the green triangle on the right side point at the signal of interest? The app automatically picks the signal to decode and, since it is designed to decode very weak signals, may lock on some noise trace, especially if the signal is dirty. There is an easy fix to that, just tap on the waterfall to lock the decoding frequency, and make sure that the triangle is yellow and it points at the signal you want to decode.

I will see if I can improve the algorithm that selects the signal to decode. For now, when testing the decoding accuracy, please lock the decoder to the signal frequency first.

[Alex VE3NEA]

Hi Ed,

Thank you for your report. I have added the Specifications section to the web page. Indeed, the waterfall displays 200-1200 Hz, and decoding occurs between 300 and 1100 Hz. This is by design, as CW pitch in the radios is almost never set outside of this range. CW Skimmer uses 3 kHz because it decodes all signals, but when only one signal is decoded, it makes sense to limit the bandwidth to the tones that are actually used in the QSO. This, in particular, allows Morse Expert to decode audio that comes out of a 500 Hz filter, which is typically used in the CW radios. The Skimmer cannot do that, it requires the filter in the radio to have a bandwidth of at least 2 kHz.

[Ed / W2RF]

Alex,

I now have CWS and ME running. The best decoding is CWS with the SDR-IQ. Second best is CWS in 3khz mode decoding audio from the 6700. Using the same audio as input the new app is actually garbling code somewhat that is copied ok by the other two methods. It does seem to filter to the selected signal when there are several in the passband. But no matter the input level the selected signal is not decoded as well as by the other two methods.

Using live sources at various speeds and quality, copy by none of the three is free of decoding errors or problems, so the comparison is somewhat subjective. Let me know if there is anything specific you want me to test.

73,

Ed / W2RF

[Alex VE3NEA]

Hi Ed,

At this point my focus is on the stability of the app. One user has reported that he was unable to save decoded text to a file. I am asking everyone to try this function and tell me if it works or not. One theory is that the user rejected the request for write permission and ticked the never-ask-again checkbox in the request dialog. I am not sure if this was the actual scenario or not, I need feedback on that. Also, a few testers have mentioned that they had to reboot/reload/restart (app or phone?), but never provided any detail. This is also something that should be addressed before I start looking into the decoding accuracy.

As far as decoding is concerned, Morse Expert is supposed to have the same accuracy as CW Skimmer since it uses the same code. Morse Expert has to solve one extra problem that does not exist in CW Skimmer, it has to select which signal to decode. To exclude this variable, please switch to the fixed frequency mode (yellow triangle) before your tests. If you still think that decoding is not as accurate as it should be, please use the Save Signals menu command to save the next 20 seconds of data, and email me the .bin file for analysis.

[ehru...@gmail.com]

Hi Alex, see inserts below. Tks, Ed / W2RF

 

From: morse-expert...@googlegroups.com <morse-expert...@googlegroups.com> On Behalf Of Alex VE3NEA
Sent: Sunday, December 13, 2020 11:15 AM
To: Morse Expert for Android <morse-expert...@googlegroups.com>
Subject: Re: Testing guidelines

 

Hi Ed,

 

At this point my focus is on the stability of the app. One user has reported that he was unable to save decoded text to a file. I am asking everyone to try this function and tell me if it works or not. One theory is that the user rejected the request for write permission and ticked the never-ask-again checkbox in the request dialog. I am not sure if this was the actual scenario or not, I need feedback on that. Also, a few testers have mentioned that they had to reboot/reload/restart (app or phone?), but never provided any detail. This is also something that should be addressed before I start looking into the decoding accuracy.

 

 

>>> decoded text is saving as expected on my Note 8, no problems to report.

 

As far as decoding is concerned, Morse Expert is supposed to have the same accuracy as CW Skimmer since it uses the same code. Morse Expert has to solve one extra problem that does not exist in CW Skimmer, it has to select which signal to decode. To exclude this variable, please switch to the fixed frequency mode (yellow triangle) before your tests. If you still think that decoding is not as accurate as it should be, please use the Save Signals menu command to save the next 20 seconds of data, and email me the .bin file for analysis.

>>> I have been using the triangle to select the signal to decode. In all cases it is bright yellow. There’s definitely a difference between my 3 test scenarios. Is DSP the same resolution in all cases? Also ME might be more sensitive to AGC pumping, but I have adjusted that as carefully as I can. I will test further and report back.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/morse-expert-for-android/47bcdd28-c3bc-4b47-afdb-5b0738663341n%40googlegroups.com.

[ehru...@gmail.com]

Hi Alex,

 

I’m working on the decoding problem. To simplify things I’m sending the sidetone from my winkey to the USB audio input on the Note 8, and I’m making sure it is selected by the triangle. My preliminary tests show that decoding degrades if dBFS > -40. I don’t see signal level in the specs. Does this make sense?

 

Thanks,

Ed / W2RF

 

From: morse-expert...@googlegroups.com <morse-expert...@googlegroups.com> On Behalf Of Alex VE3NEA
Sent: Sunday, December 13, 2020 11:15 AM
To: Morse Expert for Android <morse-expert...@googlegroups.com>
Subject: Re: Testing guidelines

 

Hi Ed,

To view this discussion on the web visit https://groups.google.com/d/msgid/morse-expert-for-android/47bcdd28-c3bc-4b47-afdb-5b0738663341n%40googlegroups.com.

[Alex VE3NEA]

Hi Ed,

Thank you for your tests! What kind of decoding degradation occurs when you set the input level above -40 dBFS? Missing chars, extra chars, incorrect spacing, etc? The decoder may miss some chars when the SNR (the third parameter on the status line) is too low. Extra spurious chars may appear when the input level (dBFS) is too high. It will help if you email me some recorded signals (using the Save Signals menu command) and/or the screenshots with decoded text.

[ehru...@gmail.com]

Hi Alex,

 

At a SNR of around 8-9dB I start getting dropout and decoding errors. The decoded chars display in a fainter font too. But from SNR just above that I get pretty good decoding. The decoding problem seems to come in at high SNR. Also it seems to take a few chars for the decoder to lock on to the “fist” and start getting chars correctly.

 

To test this I set up a white noise generator in parallel with the sidetone. Both have volume controls, so I can adjust the SNR by increasing the noise. The SNR readout seems to be on a sample and hold, similar to the speed, so it’s sort of difficult to adjust precisely, but I can work by ear too. With that setup I am able to test a lot of scenarios. Here I’m reporting just one, but they all seem similar to it.

 

With the white noise inserted in the background I sent a series of “567”. Then I dropped the noise out and repeated the series. The first set was copied correctly, but the second set was copied as “5TS7”, dropping the first dit of the 6. This seems typical of the kind of decoding error I’m seeing with real signals.

 

I’ve attached the text and signal files.

 

73, Ed / W2RF

--
You received this message because you are subscribed to a topic in the Google Groups "Morse Expert for Android" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/morse-expert-for-android/nurpTtR88og/unsubscribe.
To unsubscribe from this group and all its topics, send an email to morse-expert-for-a...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/morse-expert-for-android/b997918b-facc-445c-a80d-f0e244cc993cn%40googlegroups.com.

[Alex VE3NEA]

Ed,
This is very interesting! I would really like to see your sample signal, but I am unable to open the bin file in your attachment in my analysis software. The file is supposed to be 1024 kb in size, but it is only 480 kb. I have just tried saving signals on my own copy of the program, and the size was correct. Could you please make another record and make sure it is the right size.

[Alex VE3NEA]

Hi Ed,

Sorry for the confusion. I think I had to explain how to use the Save Signals command before I asked the testers to make recordings for me. When you click on the command, the program starts recording signals, and the message "Recording..." appears on the status panel. About 20 seconds later recording stops, at this point you can take the file to the PC via the USB interface. Please make another recording of the poorly decoded message.

[ehru...@gmail.com]

Hi Alex,

 

Sorry, here’s another try. (BTW the message I see is “Writing…” which made me think the recording process was a FIFO loop.) My current test might not be ideal, because the sidetone is full of harmonics and the noise is pure, but it’s a starting place. Hope this works.

To view this discussion on the web visit https://groups.google.com/d/msgid/morse-expert-for-android/c680d732-6c7c-49a9-9c91-f8323ed13292n%40googlegroups.com.

[ehru...@gmail.com]

Flip,

 

What’s the complete/exact name of the morse decoder you are comparing to Morse Expert? There are a ton of them in the Play store, none exactly called Morse Decoder.

 

Thanks,

Ed / W2RF

 

From: morse-expert...@googlegroups.com <morse-expert...@googlegroups.com> On Behalf Of KT9X - Lahra Flip Svare
Sent: Saturday, December 12, 2020 10:43 PM
To: Phil Deaver <phil....@gmail.com>
Cc: Morse Expert for Android <morse-expert...@googlegroups.com>
Subject: Re: Testing guidelines

 

I used mine the same way I use Morse Decoder (google play store). Just holding my phone up to the speaker. But the Morse Expert barely decoded anything, unfortunately. Morse Decoder decoded about 80% of the same signal. If this app is intended to be used with audio cable input, I did not try that, as I have a computer for that. I tried it in place of Morse Decoder, but it did not decode as I hoped. Less than 5%.

To view this discussion on the web visit https://groups.google.com/d/msgid/morse-expert-for-android/CAGN_71SUMLFrFbWywW8zV1ZPB3bejyK2fJrzzzOyrsgGD6ePLQ%40mail.gmail.com.