receiver level and low gain sensitivity change

[Alan Hopper]

Hi list,

I just noticed that somewhere between the 20151018 bit file release (the last non txdac version) and the current code the receiver levels have changed. The newer receive code gives a 6db higher power level.  Comparing new and old firmware side by side at low pga gain (0db) the new code spotted 8% more jt65/9 than the old, at higher gains the difference disappears. I did a sanity check to make sure my software is not level sensitive and it isn't so it looks like the newer firmware resolves more detail.  I first noticed this with my 16bit adc correction code which has higher levels and appears to work better still at low gains( this is with no correction).  I have not figured out what has changed in the fpga code, but whatever it is we want more of it!

73 Alan 2E0NNB

[Steve Haynal]

Hi Alan,

Interesting catch. I reviewed the commit history and don't see anything that stands out. Your predistortion code was added, there was some timing cleanup and some proper sticky settings for the AD9866. Maybe it was one of those. I'm looking forward to returning to the RTL once HL2 is done...

73,

Steve

KF7O

[Alan Hopper]

Hi List,

I've done a bit more investigation into this as I can see the effect here but no difference showed up in Claudio's noise floor tests.  I modified the synthetic test signal in the fpga, see below, to give a small signal.  On the 12bit code no signal is visible(both in SparkSDR and PowerSDR) and the data is all zeros so the signal is lost. On the 16 bit code it is easily visible at 4.6Mhz approx.  To identify where the signal is being lost I tried multiplying the signal by 16 (by shifting) before each stage of the receiver in turn.  Multiplying before the first cic was the only thing that created a signal so I guess there is an issue with this cic.  I want to learn how to do this in the emulator before going further as it is painfully slow testing this way.

In viewing these small clean signals other small spurs become visible as you tune, I guess from the cordic.  They may well be of no consequence in the real world but I do wonder if there are sweet spot cordic values that can be used with fine tuning done on the pc. I remember Steve mentioning something like this when he was looking at NCOs.

73 Alan M0NNB

      case (incnt)

            4'h0 : temp_ADC = 12'h000;

            4'h1 : temp_ADC = 12'h000;

            4'h2 : temp_ADC = 12'h000;

            4'h3 : temp_ADC = 12'h000;

            4'h4 : temp_ADC = 12'h000;

            4'h5 : temp_ADC = 12'h000;

            4'h6 : temp_ADC = 12'h000;

            4'h7 : temp_ADC = 12'h000;

            4'h8 : temp_ADC = 12'h000;

            4'h9 : temp_ADC = 12'h000;

            4'ha : temp_ADC = 12'h000;

            4'hb : temp_ADC = 12'h000;

            4'hc : temp_ADC = 12'h000;

            4'hd : temp_ADC = 12'h000;

            4'he : temp_ADC = 12'h000;

            4'hf : temp_ADC = 12'h001;

        endcase 

[Steve Haynal]

Hi Alan,

Thanks for keeping on this. There very well can be some bugs and inefficiencies in the DSP RTL. I still want to simulate all the DSP blocks in isolation and at the various expected extremes of operation, but since RTL can be fixed after the hardware is finalized I have put this off. The test you outline is definitely something we will try in simulation. If anyone is interested in diving into simulation, here is a general idea of what I'd like to do:

* Use open source Icarus or cver Verilog simulator to simulate blocks from the DSP chain, CIC, FIR, Cordic, etc.

* Cosimulate the Verilog with myhdl

* Since myhdl is in Python, it is easy to use scipy, numpy and matplotlib to generate stimulus as well as analyze and interpret results. Golden references and analysis can be facilitated with the scipy signal library.

* Waveform visualization with GTKWave.

Examples of this methodology can be found on Alex Forencich's github site (very well done, the HL2 uses his verilog-i2c IP, the myhdl cosimulations works well), the Cordic and NCO tests that I did, and the MyHDL CIC example by Christopher Felton. 

A great site to play live with Icarus, Cver and myhdl is edaplayground.

73,

Steve

KF7O

[Alan Hopper]

Steve,

thanks for all the links,  It appears rather messy to get cosimulation running on windows but getting your cordic test to run on mint in virtualbox was fairly painless.  I'll try and use that as a start for modelling the cic, time permitting.

73 Alan M0NNB

[Alan Hopper]

Hi list,

I've made a little progress on this in so much as I can explain the improvement after this release https://github.com/softerhardware/Hermes-Lite/commit/e848c52566dc25ebfd34031029a093ac743eae40 this reduced ACCWIDTH for the first cic from 28 to 27, cic.v returns the top14 bits of the accumulator so making the accumulator too long will cause loss of resolution.  To find the perfect accumulator width I want to check the output of the rx cordic i.e. are all 18 bits really used, I've tweaked Steve's tx cordic simulator to work for the rx cordic but so far it is refusing to give any useful output.

73 Alan M0NNB 

[Steve Haynal]

Hi Alan,

Did Claudio use a 73.728MHz radio for his tests? The 27-bit accumulator width only applies to 73.728MHz so he may have missed the problem you are seeing if he tested with a 76.8MHz HL2. I reviewed this today and 27 is what is expected. However, 18 + 3*log2(8) is exactly 27.0. All the other sampling rates result in a number that must be rounded up. There is not much going on in a CIC, so hard to believe something is wrong. It could still be a timing path in whatever firmware build you are using.

73,

Steve

KF7O

[Alan Hopper]

Steve,

what intrigues me is that my 16 bit receiver appears more sensitive than the the 12 bit code with the 27 bit acculmulator even though 27 bits appears to be the correct value.  I agree that it is unlikely to be the cic itself but maybe scaling somewhere along the chain. If the output of the cordic is lower than expected it would cause this or maybe something down the line is a bit out.  Both my attempts at models have the same problem, the data out is always zero, for the cic I see the output strobe toggle every 8 ticks so I know it is running, I must be doing something stupid.

73 Alan M0NNB

[Alan Hopper]

Steve,

   today's theory is that the cordic gain is not allowed for, so its fullscale output is less than 18 bits but more than 17. If I understand the cordic code correctly an extra bit is simply added to prevent overflow from the gain.  If true this means that the 14bit output of the cic is not quite big enough to capture all the detail at this stage.

73 Alan M0NNB

[in3otd]

Hello,
the measurements with Alan's code were done with a H-Lv1, using a 73.728 MHz sampling clock.

Still, I do not understand why the noise floor apparently was the same but Alan sees a better sensitivity. I wonder if it has something to do with RX IMD; does truncating one bit cause more IMD? Alan, did you use some bandpass filters in your RX test or can try to repeat the measurements with a filter?

73 de Claudio, IN3OTD / DK1CG

[Alan Hopper]

Hi Claudio,

   I am also very interested in how digital processing errors show up in external testing, All my tests were with no band pass filters.  I'm quite surprised that my two radio testing  could detect the 1 bit difference between the Steve's change in the accumulator width, but it was repeatable. The difference here was an explainable bug that Steve fixed, do your noise floor measurements detect a difference between the firmware before and after that change?  The synthetic signal test indicates there is still a small opportunity for improvement, it maybe lost in the reality of the adc noise but if my 16 bit tests aren't flawed, I think it is worth exploring. 

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

Maybe, but the original Hermes RTL has a similar pattern of bitwidths 16 input -> 22 output from RX cordic -> 18 output from cic -> 18 output from varcic. The RX cordic includes mixing the signal and I don't have a a good mathematical model for that design. I need to setup the simulation.

73,

Steve

KF7O

[Alan Hopper]

Steve,

the cordic should have a gain of approx 1.647 https://dspguru.com/dsp/faqs/cordic/ the tx code allows for this so as to use the full dac range. The rx code appears to add a bit so the gain does not cause overflow.  I might try adding a test signal to the orion code just to see, errors on the last bit with 16 bits possibly won't ever matter in the real world.  I shall persiver with the simulator.

73 Alan M0NNB

[Alan Hopper]

Hi List,

making the output of the first cic 15 bits wide instead of 14 allows the small synthetic test signal through.  After the Easter break I'll see how it compares to the 16 bit rx code which I suspect is blameless just because it uses an excess of bits all the way through.  There may be better ways of achieving the same effect such as scaling the cordic output.  

I'd like to understand the maths/thinking behind throwing away all the extra bits introduced in the cordic in the first cic, is there some noise spreading happening here?

73 Alan M0NNB   

[Steve Haynal]

Hi Alan,

I have the simulations for this working. I generate the smallest possible (1 bit changing) synthetic RF signal and feed it to the RX cordic. Output from the cordic, first CIC and second CIC can then be plotted in the frequency domain. The signal is detectable in simulation even with 14-bit width between the two CICs with worst spurs at -15 dBc. I think this is okay as with 1 bit one would expect roughly 6 dB above noise, but I am not taking into account processing gains. I am also not adding any noise or dither. Increasing the width between the two CICs does reduce spurs by 5 to 10 dB so overall may have some beneficial effect. Unfortunately, my FPGA build with a wider width did not work. I think a 16-bit instead of 14-bit width here should be the default for now. The expense is low. Once this is debugged, I will release firmware and the simulation scripts.

73,

Steve

KF7O

[Alan Hopper]

Steve,

that is interesting that you can see the signal with 14bits, I wonder where it is finally lost in my fpga. All my experiments are with the HL1 fast clock, is you simulation for this or the  HL2 cic settings. I've still failed to get any signal out of my simulation and would be very keen to see yours.  We need to create some even smaller test signals mixed with big ones so the processing gain magic can happen.

73 Alan M0NNB   

[Steve Haynal]

Hi Alan and List,

I've been sucked into simulating the receiver RTL code and creating various plots. I've probably gone a bit overboard. Instead of a long post, you can see this work on the receiver simulation wiki page. In summary, I am having a hard time seeing any difference between a 14-bit wide and 16-bit wide path from the CIC to the VARCIC once more realistic noise and dither are added. I can always see a small signal that changes at most one bit. In fact, with noise and dither, we should be able to see even smaller signals that would never change a single bit without the presence of noise and dither. Alan, please let me know the exact settings for your experiment where you saw problems at 14-bits. I will make sure to duplicate them. Also, my analysis may be rough and lacking in places. Suggestions are welcome, but I don't want to spend too much more time on this. I don't see any problems and suspect there may have been a timing error or other problem in Alan's experiment. But, the cost is low to go to 16-bits so will probably leave that in place for now.

73,

Steve

KF7O

[Alan Hopper]

Steve,

this simulator is great, it should make experimenting with the code very much faster.  I've only had a quick play but did try and recreate my simple test signal with this

    def sig(self,i):

        #return int(round( (self.scale*np.sin(2*np.pi*self.sigfreq*i*self.dt)+self.offset) ))

        return 1 if(i%16==0) else 0;

and   self.rx_freq = 4608000

      

This throws a division by zero in spectrum but once fixed it shows the same loss of signal in the first cic that I see on the hardware. It maybe that this test signal is not important in the real world but the simulator would suggest it is not a timing error and mathematically the cordic gain issue makes sense.  I shall play more and hopefully get a chance to run some more on air tests.

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

This strikes me as a pathological case. First, to see this type of waveform at the output of the ADC I imagine you'd have a slight DC bias at the input such that only the tips of the sinusoid positive peaks trigger a reading of 1. Also, this synthetic signal is a precise simple integer fraction (1/16th) of the sampling frequency. In the no noise simulations, I do not see any signal at 14 bits but do see a signal at 16 bits. If I add just a touch of noise, noise = np.random.normal(loc=0.0, scale=4.0), I see the signal at both 14 bits and 16 bits and they are both about the same amplitude over the noise, maybe a slight edge to the 16 bits simulation. If you consider the theoretical gain of the Cordic (1.647 as you referenced) plus the expected processing gain from the first decimate by 8 CIC (9.0 dB) then 14 bits is just barely enough. An extra bit or two for guarding may help.

[Alan Hopper]

Steve,

I agree it is an extreme and unrealistic case but one I think the digital side should cope with. I'm not sure it explains the on air differences I see, maybe the improved cordic spur reduction is more significant (reduced IMD?), or maybe my testing is flawed.  I used the simulator to confirm the cordic gain is 1.647 (+extra bits), this is of course the gain of the magnitude of the IQ signal, for the test signal the maximum I and Q values are lower still as the phase never lands exactly on 0/90 etc.   I'm guessing there is an optimum relationship between the number of extra bits used in the cordic and the bit width after the cic, maybe we can save some gates here.  I shall pit your new code against the full 16bit code on air.

Thanks again for the simulator, It has already saved time on something else.  btw I have been using 'visual studio code' on linux as an editor, the 'open folder' option gives easy access to files and there is syntax colouring for python and verilog, probably not the tool for linux lifers but nice and familiar if you live in the visual studio world.

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

With signal generation of 1 if i%16 == 0 else 0 there is no signal after the CIC at 15-bit width. If the synthetic signal is shifted to 1 if (i+3)%16 == 0 else 0 then the 15-bit width will produce a signal after the CIC. A 16-bit width always appears to produce a signal after the CIC. I'd like to eventually model the final polyphase FIR but those use Altera .mif files and I need to convert those to pure Verilog to simulate. Simulation time will increase. I'd also like to figure out how to apply an impulse or chirp to see the response of the entire chain. 

What else are you using this simulation for?

73,

Steve

KF7O

[Alan Hopper]

Steve,

with an extremely large test signal my 16 bit receiver was going wrong, I was able to discover that it was an overflow in the fir by using the simulator to eliminate the cics from blame. I'm still not sure why this overflow exists but have fixed in in the version on github.

 I have also been working on using the cic and fir from the protocol 2 (formally known as 'the new protocol') code. I have modified the cic so the first comb and second integrator and comb run sequentially to reduce gate count, it is early days but I can fit 5 receivers without fir or 3(nearly 4) receivers with fir in the cv. The protocol 2 fir can be made to use half the number of multipliers if we don't use  the higher bandwidths (768 and 1536k) and maybe half again if the symmetry of the fir is exploited.  The simulator is going to be a great help here.  I had been wondering how to cope with the .mif files.  I want to add a option in a debug build to be able to download test synthetic waveforms to the hardware.

73  Alan M0NNB

[Steve Haynal]

Hi Alan,

Sounds like fun! Good luck with packing all the receivers!

73,

Steve

KF7O

[Alan Hopper]

Steve,

I've been running the full 16 bit code vs the 12 bit code with 16bits out of the first cic for the last couple of days, after 12000 jt spots the difference is now less that 1.4% (in favor of the full 16bit code) this is at the top limit of difference I see with identical firmware so probably irrelevant.

All these tests have been done with half duplex as only one of my radios supports full duplex.

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

Thanks for the report.

73,

Steve

KF7O

[PA3GSB]

Hi guys

Trying to do the simulation... real nice option.
Running ./test_rx2IQ.py test.pkl -sim

results in:

 File "./test_rx2IQ.py", line 272, in test_bench
    sim = Simulation(self.bench())
NameError: name 'Simulation' is not defined

which component do i miss?

73 Johan pa3gsb

[Alan Hopper]

Johan,

I had to uncomment this line near the top

#from myhdl import *

73 Alan M0NNB

[PA3GSB]

Alan,

Tnx

I have it working now.

Now learning....

73 Johan PA3GSB

Op donderdag 18 mei 2017 21:55:11 UTC+2 schreef Alan Hopper: