More FSK fun

[David Woodhouse]

I just submitted a couple more watchman traces, this time at 1MHz —
which seems to be sufficient for the emonTx device too:
https://github.com/merbanan/rtl_433_tests/pull/71

The spare watchman device I bought for playing with, which is sitting
in my study right by the receiver, is fine. This is gfile151.data.

The one that's actually installed outside on the oil tank is not being
decoded properly. This is gfile023.data.

The signal is a little bit noisy, but it's not *that* bad. Here's
Audacity showing the problematic signal in both I/Q and FM-decoded
form, along with the good one for comparison at the bottom.

The PR also includes a version of my original fskdemod.c hack, hacked
further to make it decode the noisy signal. I did two things — firstly
I eliminated spikes by working with a *mean* value over 8 samples, and
secondly I tweak the 0/1 threshold so it's *lower* when we're in the
middle of a 1 bit, and *higher* in the middle of a 0 bit. Basically
making it harder to change from 0 to 1 and vice versa, and again coping
with noise.


--
dwmw2

[Tommy Vestermark]

The FSK demodulator should be fairly resistant to these noise spikes. The FM signal is low-pass filtered (bandwidth 20% somewhat equal to 5 samples averaging) and the FSK demodulator will suppress pulses below 10 samples. But beware that "oversampling" the signal defeats these filter mechanisms and makes the frequency deltas relatively smaller. It also increases the general noise level and opens the bandwidth for more spurious interference. So in general it is not advisable to increase the sample rate unless the signal has very short pulses or high frequency deltas necessitating it.

However it turns out the gfile023.data decodes quite nicely, when using -l 1000. In this particular case the issue is the carrier detection not triggering the FSK demodulator in the first place. The level estimates indicate there should be enough signal/noise ratio, so I will look into that. We might also utilize the FM signal to improve carrier detection - but that is a new adventure :-)

The hysteresis in frequency threshold was removed in the rework and should be somewhat redundant to the short pulse suppression. I might add it back in, if we see some real benefit from it.

I don't think this should hold off merging the changes to the FSK demodulator.

/Tommy

[Tommy Vestermark]

OK, I looked at the troublesome oil_watchman files:

• 05/gfile023.data would not be detected because the leading edge of the carrier rises slower due to the higher sampling frequency. As a result the low level (noise) estimate would rise too fast and artificially increase the detection threshold ahead of the signal. There is no reason for the noise estimator to be that fast, so I have slowed it down.
• 05/gfile239.data would not be detected because of excessive noise sometimes causing the carrier detection to declare end of pulse. I have implemented suppression of short gaps (like for the FSK decoder).
• 05/gfile455.data is simply a malformed file with no lead-in.
• 02/gfile001.data is also a malformed file with a strangely saturated lead-in.
  

The changes can be found here: https://github.com/merbanan/rtl_433/pull/272. It now decodes all the other oil_watchman files - maybe the malformed ones should be removed?

I think hunting all these files have made the FSK demodulator and auto level detection more robust - but you don't have to go out of your way to find more troublesome files straight away ;-)
/Tommy

[David Woodhouse]

On Sat, 2016-01-23 at 06:46 -0800, Tommy Vestermark wrote:
> 05/gfile455.data is simply a malformed file with no lead-in.

Do we blame the level detection for that?

> 02/gfile001.data is also a malformed file with a strangely saturated
> lead-in.

Not one of mine; not sure how that happened.

> I think hunting all these files have made the FSK demodulator and
> auto level detection more robust - but you don't have to go out of
> your way to find more troublesome files straight away ;-)

Thanks again for your excellent work on this!

It's now fairly reliably decoding the signals from the oil tank in the
garden, from fairly much the opposite end of the house. I think it's
good and I'll try to stop torturing you, as requested :)

-- 
dwmw2

[Tommy Vestermark]

On Saturday, January 23, 2016 at 8:42:36 PM UTC+1, David Woodhouse wrote:

On Sat, 2016-01-23 at 06:46 -0800, Tommy Vestermark wrote:
> 05/gfile455.data is simply a malformed file with no lead-in.

Do we blame the level detection for that?

 

I don't think so. The Pulse Detector/Analyzer '-A' is completely seperate code from the Analyze Mode/Test signal save '-a -t'. Even though it is somewhat redundant and the functionality should probably be merged - it has been very valuable in the evolution of the Pulse Detector that the test signals have been generated by "unbiased" code. :-)

 

It's now fairly reliably decoding the signals from the oil tank in the
garden, from fairly much the opposite end of the house. I think it's
good and I'll try to stop torturing you, as requested :)

Good it is working more reliably. You shouldn't stop finding new issues. However it would be nice with some period of code stability ;-) Also we might approach some tradeoffs between cost vs gain or specialized tuning versus genericness. Off course it is also likely we might hit a bug or two in the code as is... Signals that can be detected by setting manual level or otherwise look OK by inspection are always interesting.

You are welcome and thank you too.
Tommy