Those are interesting results. Thank you very much for all your investigative work. A few thoughts below.
Regarding the "finger fix" for sensitivity: your finger would have been adding resistance, capacitance and inductance in unknown quantities, and to various points on the board. It might be very difficult to determine an equivalent circuit. But the fact that sensitivity improved with the removal of the 51-ohm series attenuator resistors might be a clue as to what was happening. Perhaps the finger load was coupling more clock signal across the attenuator and into the SA605.
The expected impedance from the Si5351 is vague in the documents I've found, but it is supposed to be able to drive something between 50 and 85 ohms. But the signal amplitude might drop under those loads. The input impedance into the clock inputs of the SA605 are also vague. They likely vary with frequency just like the RF input pin does. One application note indicates that the input impedance might be around 10k, so that's what I assumed when calculating the resistor networks. All that is to say that experimentation is almost certainly needed: after all, we aren't exactly following any of the circuits provided by the manufacturer, and we don't have SA605 models (even poor ones) with which to test our designs.
You have established that it should be possible to achieve ~1uV sensitivity with the SA605 alone. That is within ~14dB of our goal of 0.2 uV. So our front end should have more than enough gain (provided that it all works!) to get us to our goal. And since we can totally remove all the gain from the front end, and attenuate the signal by more than 10dB by turning off the pre-amp and gain block, we shouldn't have to worry about there being too much gain in the front end.
The extended range of the RSSI is quite interesting. In whoopee mode, the user will get the full benefit of the extended RSSI dynamic range. AN1996 provides a circuit that is supposed to extend the RSSI range even further. But perhaps by the time the RSSI voltage is maxing out, the RF detector circuit we added will be detecting the RF signal directly and will provide an alternative RSSI source that works to much higher signal levels.
Based on your testing I will remove the option to externally bias pin 10, and leave only the option to bias pin 4. Based on your numbers, I think you are seeing at least 40dB of attenuation available by that method. That is good, but maybe not as much attenuation as we'd like. Also, I don't like its effect on the RSSI voltage.
There are still a couple of other options for deafening the SA605: biasing pin 1 (and/or 2); or coming up with the equivalent of a variable R212 gain killing resistor. Both of those options might have the benefit of not up-biasing the RSSI. I might look into adding pads that will allow us to experiment with those two options - especially since it looks like biasing pin 4 alone might not provide all the attenuation we want.
Thanks again for all your help!