Question on RMS-based integration time planning for HI observations

[Tiago Baroni]

Dear all,

I am at a very early stage in amateur HI radio astronomy and have been trying to move away from the usual heuristics I often see (fixed numbers of integrations, “rule-of-thumb” observing times, etc.), which I find hard to justify quantitatively when planning observations with small instruments.

As an exercise, I have been working through a simple, iterative planning approach based on the measured RMS of pilot observations and the radiometer equation. In essence, the idea is to (i) measure the RMS after baseline removal in line-free regions, (ii) compare the observed RMS with the expected thermal RMS to check whether the system is still in a thermally dominated regime, and (iii) only scale integration time according to the 1/τ​ law while that condition holds. When the RMS stops decreasing efficiently and appears to asymptotically approach a constant value, this is treated as an instrumental/systematic floor and used as a practical stopping criterion, rather than continuing to integrate blindly.

I am not claiming any novelty here; this is mainly an attempt to formalize, in a transparent way, decisions that are often made implicitly. I would very much appreciate feedback from the community on whether this conceptual approach makes sense in practice, what assumptions might be too optimistic for small-dish / SDR-based setups, and whether there are known statistical or instrumental pitfalls (e.g. correlated noise, baseline effects, RFI handling) that tend to invalidate this kind of RMS-based planning if one is not careful.

Comments on how applicable (or not) this framework is across different receivers, backends, or observing strategies would also be extremely helpful. I am very open to corrections and criticism, as this is primarily a learning exercise.

Best regards,
Tiago Baroni

[Alex P]

Hello Tiago,

In principal, your idea is valid ....

BUT, since you are using an SDR , why integrate a Doppler Velocity Shifted Spectrum into a single RMS value ?

I am sure that will stabilize in a very short (5- 10 sec ? ) time period, but capturing data at a rate faster than say 1/10th to 1/20th the beam width of your antenna 

may yield huge numbers of samples per day, but with little useful value, and if observed as a spectrum : quite noisy .

This data is from a 1m class dish and 300 seconds  ( 1.25 degrees of RA drift ) integration per sample 

Regards,

Alex Pettit

https://github.com/AP-HLine-3D/HLine3D

[fasleitung3]

Hi Triago,

Your approach is perfectly fine. In fact, this is exactly what we are asking the master students to do which come to us for a lab course:

As a first step, they measure the system temperature by determining the rms noise of the baseline in comparison to a calibration source like S7. Then they are supposed to observe the hydrogen spectrum of another galaxy. For this, they need to determine what integration time is needed for a specific desired SNR. Taking the brightness temperature of the galaxy from literature, they can do this by applying the radiometer equation. So indeed, what you are proposing is done in practice.

Of course, there are certain limits and caveats which you have also pointed out: RFI will contribute to the baseline rms noise and typically will not decrease with integration time. So in reality the system sensitivity will be degraded by this and a desired SNR may not be achieved even with long integration times. Another factor is the system Allen time. This, however, is more of a theoretical nature as modern digital systems typically have a very long Allen time.

Best regards,

Wolfgang

--

[b alex pettit jr]

Hello Wolfgang,

What sized system are you using ? 

 "We" tried this with my 1m a few years ago but with ambiguous results as the S7 calibration region is quite small 

Thanks,

Alex

================

[Tiago Baroni]

Hello Alex, hello Wolfgang,

Thanks for the helpful comments. Let me briefly clarify one point.

The RMS in my approach is not meant to represent a Doppler-shifted spectrum or to extract scientific information from individual sub-integrations. It is used purely as an operational sensitivity metric, computed after combining sub-integrations, baseline removal, and restricted to line-free regions, with the sole purpose of integration-time planning while the system remains approximately thermal.

Alex’s point about oversampling relative to the beamwidth is absolutely valid for sky mapping and independent spatial sampling. My use of short sub-integrations is instead for quality control (RFI rejection, gain stability), with RMS evaluated only after their combination.

Wolfgang’s remarks align well with this view: baseline RMS and the radiometer equation are useful planning tools, but in practice residual RFI, gain drifts, and other systematics quickly set an operational floor where further integration becomes inefficient.

As a beginner, my goal is mainly to learn how to identify this transition, rather than assuming that longer integration always helps. I appreciate the feedback and am happy to hear further suggestions, especially regarding common pitfalls in small SDR-based systems.

Best regards,

Tiago Baroni

[b alex pettit jr]

Hello Tiago,

FYI, these are plots of the same data 

Power Sum of each sample

Doppler Spectrum of each sample  ( FFT = 512 freqs )

Regards,

Alex Pettit