This is mostly conjecture based on my understanding, but to get better dynamic range we would have to sweep each bin in multiple passes. The first pass would be with the preamp disengaged; then a second pass could be made with the preamp engaged for the frequency ranges where it can be safely engaged without causing ADC clipping. A calibration step to characterize the preamp itself would also be necessary. So I'm thinking it's a software problem, albeit making the algorithm much more complicated than we might initially suspect. Maybe more sophisticated software already can do this.
I agree with you though that we could probably see better than 40dB of dynamic range without having to resort to trickery with the preamp. I think steve was seeing about 52dB, so I do not know what is going on with your setup. Perhaps this is a limitation of the drive power being unable to fully saturate the ADC (due again to the existing software being written for the capabilities of the hermes/angelia/orion hardware). It may also have something to do with the bit errors that you and Steve identified in the first release of the full duplex code. I would have expected we could get closer to 70dB by default, and perhaps a lot more from process gain even without employing the preamp. Since we have an easily accessed onboard attenuator and preamp, I would think the Hermes-Lite may eventually be more capable than the Hermes as a VNA if the software is extended.
It looks like HamVNA runs either with the "attenuator" disabled or optionally enabled at the user's request. It seems to run with drive fixed at 0, and it does not adjust the step attenuator value that I can see. There is some kind of funny interaction with the attenuator/preamp/atten value that I do not understand. It detects the ADC overload condition reported by the RTL should it occur.
I am not sure whether these baselines used by HamVNA are appropriate for Hermes-Lite. It appears that in VNA mode the hermes-lite drive controls are remapped by the RTL to limit the maximum Tx power to about 15mW, but with drive=0 perhaps we are still under-driving. It also appears that in VNA mode the RTL sets some limits set on the preamp; maybe to prevent possible damage?
Here is where HamVNA initializes the Hermes. It sets attenuator=off, preamp=on, vna=on, drive=0
Here is where HamVNA enables the attenuator value when the checkbox is marked.
I finally had time to build my 1.22 this weekend, so I am looking forward to experimenting with the VNA now as well. First step is probably to see if I can get the tools to compile HamVNA and then dust off the part of my brain that hopefully still holds some Pascal syntax. Is any of this Delphi stuff freely available?
73, John K5IT