adc correction

[Alan Hopper]

Hi list,

following on from this discussion https://groups.google.com/d/msg/hermes-lite/0P18xh-KaXQ/dy2jp7dsAAAJ I have had a play at trying to linearize the adc.  At the fpga end I have a lookup table between the adc and the receiver code that allows arbitrary correction and has 16 bit output. I have increased the rx bit width code to 16 bit with a mix of hermes lite and hermes code.  The lookup table values are sent in the unused audio packet. This side all works and fits in a CV board.  I can generate lots of imd by sending a very non linear table!

At the pc end I have code to analyse the  raw adc values sent in the wideband packet. The idea is to feed a signal with a known probability distribution function into the receiver at a level that will clip.  A histogram of adc levels is collected over many millions of samples.  I have code that will work with Gaussian noise or a sine wave and calculate the lookup table by comparing the measured histogram with the predicted distribution.  The clipping values are used to work out the section of the probability function that covers the adc range.  This all seems to work with simulated data.

I tried first with noise from the antenna, there were glimpses of believable results but nothing repeatable.  A cheap ebay noise source was repeatable but nowhere near Gaussian, I guess I was effectively measuring the non linearity its amplifier.  I have not tried with a sine input but am not sure I have a source of high enough quality.

If anyone has a high quality noise or sine wave source and fancies doing some tests please give me a shout, I'm afraid the software is windows only.

I'm looking for ideas for a cheap calibration source.

It may be possible to use a combination of the cheap noise source to correct the close in bit by bit errors and then Claudio's suggestion of measuring imd and minimising it to correct the errors over the full range of the adc.  My next step is to setup something so I can measure imd.

I have no idea if this will result in any improvement but trying was irresistible.

73 Alan 2E0NNB

[in3otd]

Hello Alan,
nice to see you already have something working :) I was wondering if the TxDAC could be used for generating the needed sinewave, with an additional bandpass filter to remove the noise/spurs.
A good sinewave source would of course be a crystal oscillator but  many oscillators have a square wave output so you'll need to filter out the harmonics. You'll need to compute how much harmonic suppression is needed to obtain a good enough PDF, hi.
Here my bench is linux only so I cannot test your SW at the moment, not sure when I'll be able to clean it up a bit to make room for another PC...

73 de Claudio, IN3OTD / DK1CG

[irbsu...@yahoo.co.uk]

Hi Alan,
Very interesting work, maybe I can help with some tests, look forward to discussing it more at the convention.
Andrew
G4XZL

[Alan Hopper]

Hi List,

I managed to get a measure of the adc linearity with my cheap noise source by smoothing the measured histogram and subtracting it from the un smoothed one. The idea is that whilst the noise source is not perfect it does not have sharp changes in its distribution so sharp changes are assumed to be adc errors. It is described in a link that Robert 

G3WKU sent me here http://www.control.isy.liu.se/research/reports/2001/2400.pdf  . It won't detect smooth errors caused in the pga.  The graph is the deviation from a straight line. The maximum error appears to be around 1/4 of a least significant bit so not much. In applying the correction I have yet to detect any advantage but It is early days.  This was a first attempt so it could be completely wrong!

Andrew I'm looking forward to meeting, it would be interesting to compare this result with a test with a quality noise source or sine wave.

73 Alan 2E0NNB

[Alan Hopper]

Hi List,

the attached file has jic files for cv, cva9 and sdk with the experimental adc correction in, the changed source files are also included.  I have only tested the cv and cva9 jic files.  This firmware won't receive without modified software as the correction tables default to being empty. Version 0.140 of my software http://www.ihopper.org/radio/ has some tools to allow experimentation.  On start up the radio should work as normal but with higher levels, a linear correction table is loaded by default.

The radio settings page (...) has the following controls

Load adc correction - this allows selection of a .txt file from disk and sends it to the radio.

Clear adc correction - this loads a linear (no correction) table to the radio.

Grab histogram -  this builds a histogram from the number of blocks of adc data specified in the text box next to the button. 40 is a good starting point to check the levels, a crude histogram will be displayed underneath, for the rest of the process to work the levels should cause clipping at both ends.

Once the levels are ok more blocks should be used to get smoother data, say 10000.  When the grab is complete 4 files will be written to c:\users\##your name##\appdata\roaming\m6nnb\radio\settings.

xxxxxx_hist.txt this is the raw histogram data.

xxxxxx_lut_noise.txt this is a correction table that assumes the test signal was Gaussian noise.

xxxxxx_lut_sin.txt this is a correction table that assumes the test signal was a pure sine wave.

xxxxxx_lut_smooth.txt this is a correction table that is calculated by smoothing the test signal.

xxxxxx is a mix of the radio mac address and the time.

The histogram file could be used to try other means of generating the look up table.  This is all very experimental, I have yet to detect any gains from using it.  This release is really for those with better test kit than me who fancy some experimentation.

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

Thanks for all you work on this! I've installed your firmware on my CVA9 and am running your latest 0.140 software on my tiny Liva PC. Your NTP frequency calibration, PSK decoding, computation spreading, sound off, and ADC correction functions are all quite nice. How many receivers does the CVA9 firmware have? I can't seem to go above 3.

73,

Steve

KF7O

[Alan Hopper]

Hi Steve,

I'm afraid I only built the cva9 firmware for 3 receivers, I'll build a bigger one today. They are all half duplex. I had to limit the sdk to 2 receivers, if it proves to be useful, the table size can be shrunk once we know the maximum error.  I have no idea if the 16bit receiver code is properly optimised for word length etc.

73 Alan M0NNB

[Alan Hopper]

Steve,

cva9 hd built for 32 receivers.

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

Thanks for the bit file. I did try it out but it didn't work. I went back to other 32 receiver bit files but they didn't work either. I made a change to my CVA9 power supply and I think it has problems supplying current with that many receivers. I need to debug the power supply.

73,

Steve KF7O

[Alan Hopper]

Hi list,

I've not got any further with this due to lack of proper test kit. I need a clean sine wave source for calibration and also would like to be able to measure IMD to look for improvements (and also for dither experiments) I'm too mean to buy a pair of high quality signal generators so am thinking of building a pair of fixed frequency generators to do both jobs. Does anyone know of an existing suitable design?  My rf design skills are sadly lacking. Am I better off using a square wave crystal oscillator module and filtering it or building a traditional crystal oscillator circuit (with maybe less filtering)? For adc calibration use the amplitude stability is important, what spec should I be aiming for for IMD measurement use. All ideas welcome.

73 Alan M0NNB

[in3otd]

Hello Alan,
are you sure you want a *fixed* frequency generator? Once you do all the measurements at one frequency you *will* be curious to know what happens at other frequencies too, hi.

I have this signal generator based on the Si570: its phase noise is quite good but the output is a square wave, so you need to filter it. I have no data on the output level stability. There are many similar generators using the Si570, like the one here which someone used to build a two-tone generator.
For a fixed-frequency generator, a crystal will be even cleaner: this design uses also an additional crystal at the output as a filter.

73 de Claudio, IN3OTD / DK1CG

[Alan Hopper]

Claudio, 

you are quite right (I would be curious), thanks for the Si570 links, I think I'll try that approach as it appears proven.  Once I have made some filters I might try a simple crystal just for my own education.

I did get a believable looking calibration using a filter on a second hermes lite as a source but had no means to measure any rx improvement, a side by side test showed no difference but as adding a rx bandpass filter here also shows no difference I'm not sure that tells us much.

73 Alan M0NNB

[Alan Hopper]

Hi Steve,list

the fpga code uploaded earlier in this thread works with the later higher speed clock.  I produced a version for Andrew G4XZL for the older slower clock as he has kindly been doing some testing and it appears to be clipping. I wonder if anyone has a quick guide to correctly set the bit widths for the cics as the receivers are now given 16 bit rather than 12 bit inputs. I don't have a radio with the slow clock so can't test.

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

I would grep the RTL for use of the parameter "CLK_FREQ" and other parameters which are set by CLK_FREQ (lines 100-125 in hermes_lite_core.v) and make sure everything happening for the slower clock matches what you've done and want for the faster clock. Sorry, but this is the quickest solution for the current RTL.

73,

Steve

KF7O

[Alan Hopper]

Hi List,

the rtl source code for this is now in its own branch on github here https://github.com/ahopper/Hermes-Lite/tree/ADC-Correction

73 Alan M0NNB

[in3otd]

Hello Alan,
the ADC correction table is still empty at power up in this version? Since I have now SparkSDR working (...) will it work if I connect to the H-L, use "Clear adc correction" to load a linear table, disconnect SparkSDR and then use Quisk to receive? Will the correction table be affected by disconnecting/reconnecting?

73 de Claudio, IN3OTD / DK1CG

[Alan Hopper]

Claudio,

I'm not quite sure, it depends what Quisk sends in the audio samples, it might corrupt the table but you might be lucky.   Yes the table is still empty at power up.  I'll look at filling it somehow, you could probably bypass the table lookup and just put the 12 bit data in the high bits of the 16 receiver input around line 876 in hermes_lite_core.v. I might just get a chance this evening.

73 Alan M0NNB 

[in3otd]

ok, you mean it will be enough just to uncomment these lines and comment out the ones below? I can try later to see how it works.

73 de Claudio, IN3OTD / DK1CG

[Alan Hopper]

Claudio,

yes I think so, I'd forgotten doing that.  I'm sorry I don't think I'll get a chance to do anything myself tonight. If you happen to be a Italy rugby supporter, great match.

73 Alan M0NNB

[in3otd]

no problem, Alan, I'm rebuilding the FW with the changes described above; hopefully I should be able to have some test running this night.

73 de Claudio, IN3OTD / DK1CG

[in3otd]

Hello Alan,
I've measured the noise floor of your ADC-Correction FW, with the modifications described above to actually bypass the correction; the results are practically identical to the other measurements. The (relative) gain is higher but this I think is expected. Not sure why this FW gave better results in your tests.

Here is the measured noise floor for the various gain steps:

to be compared with the original results here.

The relative gain measured has the usual full-duplex ripples and is higher than the standard FW:

[Alan Hopper]

Claudio,

thanks for that, I'll recreate my test and see if I can find an explanation.

Alan

[Alan Hopper]

Hi List,

I've finally pulled together some kit to allow me to generate a clean test signal and measure imd. I have two si570 ocillators, ( a http://www.sdr-kits.net/Webshop/products.php?34&cPath=6 and the clock section from a softrock rx ensemble).  To filter them I have 2 qrp labs 30m bandpass filters and two qrp labs 30m lowpass filters.  

I used a single oscillator with all 4 filters to generate a correction table using an assumed sine wave probability distribution for the test signal.

The two traces were created with the same settings but a few hours apart so repeatability is not bad.  The error is scaled to 16 bit so the maximum measured error is only just over +-1bit of the adc.  This was with a lna gain of 24db, everything was screened and the generator was battery powered and disconnected from usb.  

My imd measuring setup is waiting for a couple of bits but in a cobbled together test I did see a drop of 13db of im3 peaks, I would not read too much into this as it was a very rough test, but it is motivation to test further.

I don't think this simple lut correction will help with the imd caused by the digital outputs of the adc identified by Claudio, I wonder if correction is possible for this as we know the delay from adc input to digital output so a correction could be applied based on which lines are switching at the time a sample is taken.  The lut also does not allow for any slew rate limiting in the input path. 

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

Thanks for your work. It sounds like there are some interesting possibilities. I will have to dig in deeper once the PCB is done and more HL2 are out in the wild.

73,

Steve

KF7O

[Alan Hopper]

Hi List,

I finally have a reasonable imd test setup. A quick test with the correction table above produced this

The close in peaks drop by about 17db but the next two out pop up by about the same amount.  If I use an lna gain other than the one(+24db) that I used to generate the table all the peaks go up, so it appears a table will be needed for each gain should  this prove to be of any value.  Taking up a suggestion by Claudio  I'm tempted to add a manual set of power series terms to the correction  to compensate for errors in my calibration signal.

In looking for a calibration source I found this interesting note https://www.intersil.com/content/dam/Intersil/documents/an10/an1088.pdf

73 Alan M0NNB

[Alan Hopper]

Hi List,

I've added the ability to apply power series terms from x^2 to x^7 in the adc correction tab in Spark sdr http://www.ihopper.org/radio/hlradio_0_150.zip .  The manual corrections can be applied to a table generated from a calibration signal or to a default linear table.

 Tweaking the x^3 and x^5 terms I was able to reduce 3f peak by the same 17db without the increase in the 5f peak. The correction works for different input levels but I have yet to test at different frequencies (more filters needed).

It appears my calibration signal or maths need some work to avoid the manual tweaking.

73 Alan M0NNB