code for automatic gain functions?

[Kristoff Bonne]

Hi,


I'm currently playing around with tlb (thelinkbox, echolink application)
to make it accept raw PCM input.

The goal is to use a RTL DVB-T stick as receive-only input for it, using
a FIFO named pipe as communication path. Sofar, it seams to work ok,
except that the audio of "rtl_fm" of the demodulated audio is pretty low.
As the RTL-dongle is not a real audio-device, I cannot use the mixer to
correct this, so I probably need to write a small "automatic gain"
application to fix this.



Does anybody know if there are any open-source implementations of an
automatic gain function that processes a raw audio-stream (signed 16 bit)?



73
kristoff - ON1ARF

[Tony Langdon]

Do you need AGC, or will a simple "set and forget" gain control do the
trick?

--
73 de Tony VK3JED/VK3IRL
http://vkradio.com

[Kristoff Bonne]

Tony,

Well, a "set and forget" value would be nice to start with but I think I
can code that myself. :-)


It's the AGC that is the more difficult part. I think it should be
something like this:
- apply a minimal "squelch" treshhold
- if the threshhold is exceeded for a sufficient number of samples,
calculate the average audio-level over (say) the last second
- calculate a amplification-value based on this
- multiply all incoming audio with that value


The more difficult part will probably be on how fast to make the AGC
addapt itself. Also, as rtl_fm does not seams to apply a LPF everything
below 300 Hz, I probably also need to add that too to that the AGC does
not get confused by that.


Another thing is that I have just noticed that the demodulated audio
from rtl_fm has a terrible DC offset. My guess it is related to drift
off the tuner in the dongle (which varies over time as these tuner are
quite sensitive to temperature). I probably will need to get rid of that
before any processing off the audio can be done.
Grr!!! Why do the details of implementing stuff aways make things so
more difficult then first thought! :-(


BTW. Thanks for the hint on using the eventmanager in tlb to set up a
connection between two TLB devices on startup. It surely is usefull!



73
kristoff - ON1ARF

[Steve]

Another version:

http://www.mathworks.com/matlabcentral/fileexchange/11202-automatic-gain-control

It is written in C.

[Kristoff Bonne]

Hi Steve,


Well, I had already thought about using gnuradio for this but as the platform I use for this (a pogoplug refleshed with generic archlinux) does not support GNUradio at all, I didn't think much further about this.

You are right. The sourcecode of the gnuradio blocks is probably a good source for this, if not for the code itself, but at least about some ideas on how to implement it.


Thanks!

73
kristoff - ON1ARF



On 12-10-13 16:26, Steve wrote:

Hmm, gnu radio has some C++ code: http://gnuradio.org/doc/doxygen/agc_8h_source.html

It has both a float and a complex version in the file.

Some old code I have laying around which is passed 8 kHz 16-bit PCM samples divided by 32768.0 to produce -1.0 - 1.0 float values which are then run through the FIR filter before going to an agc algorithm:

    private static final double LAMBDA = 0.002;    // .02 ??

    private static double agc_hold;

   
    /*

     * Automatic Gain control

     */

    private Complex agc(Complex in) {

        double magnitude = in.mag();          // magnitude = sqrt(real * real + imag * imag)

        double h = (LAMBDA * magnitude) + (1.0 - LAMBDA) * agc_hold;

        agc_hold = h;

        return in.times(0.001 / Math.sqrt(h));

    }

I've never used this function, but it is laying around in my toolbox.

--
You received this message because you are subscribed to the Google Groups "digitalvoice" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice...@googlegroups.com.
To post to this group, send email to digita...@googlegroups.com.
Visit this group at http://groups.google.com/group/digitalvoice.
For more options, visit https://groups.google.com/groups/opt_out.

[Tony Langdon]

On 12/10/13 10:47 PM, Kristoff Bonne wrote:
>> Do you need AGC, or will a simple "set and forget" gain control do
>> the trick?
>>
> Well, a "set and forget" value would be nice to start with but I think
> I can code that myself. :-)

My point was to ask the question of whether you were overthinking this.
:) How many of us use AGC on the audio between FM links?

>
>
> It's the AGC that is the more difficult part. I think it should be
> something like this:
> - apply a minimal "squelch" treshhold
> - if the threshhold is exceeded for a sufficient number of samples,
> calculate the average audio-level over (say) the last second
> - calculate a amplification-value based on this
> - multiply all incoming audio with that value
>

Something like that.

>
> The more difficult part will probably be on how fast to make the AGC
> addapt itself. Also, as rtl_fm does not seams to apply a LPF
> everything below 300 Hz, I probably also need to add that too to that
> the AGC does not get confused by that.

Actually a constant tone would limit the range of the AGC, so that would
help, if anything. Or you could be clever and once you'd got a sense of
the overall audio level for the transmission, look at the energy below
300 Hz, and if there was a tone, hold it at constant level to avoid AGC
pumping. :) Actually I thought there was already a 300 Hz HPF in tlb.
Have to double check that one. Though if you put your AGC after that,
you could make it selectable for all inputs.

>
>
> Another thing is that I have just noticed that the demodulated audio
> from rtl_fm has a terrible DC offset. My guess it is related to drift
> off the tuner in the dongle (which varies over time as these tuner are
> quite sensitive to temperature). I probably will need to get rid of
> that before any processing off the audio can be done.
> Grr!!! Why do the details of implementing stuff aways make things so
> more difficult then first thought! :-(

A HPF would eliminate this as well as stray CTCSS tones (DC is 0 Hz
afterall :) ).

>
>
> BTW. Thanks for the hint on using the eventmanager in tlb to set up a
> connection between two TLB devices on startup. It surely is usefull!

Works a treat, and you're welcome. :)

[jdow]

Neither one seems to show the fast attack and slow decay properties
you would like to see. I guess that is an exercise for the student
one the proper function of AGC is figured out.

{^_-} Joanne/W6MKU

[Tony Langdon]

On 13/10/13 10:43 AM, jdow wrote:
> Neither one seems to show the fast attack and slow decay properties
> you would like to see. I guess that is an exercise for the student
> one the proper function of AGC is figured out.

You could even try something like fast attack and hold, since the audio
level of FM transmissions tends to be fairly constant, with the gain
resetting when either (a) the transmission ceases (as determined by the
squelch), or (b) the audio level falls below a minimum threshold for a
period of time (if you can't get access to the squelch signal). But
with tlb, I'd assume you need some sort of squelch function so the
system is not tied up in between overs.

[jdow]

That strategy is indeed better yet - even for SSB. The maximum ALC level
for the last couple minutes should be remembered. It should only be
changed if the level is bumped up or of the speech is continuously 1dB
or more below the set power level. Anything else tends to produce
considerable splatter, particularly when ALC is used as a power output
control at other than full power. And ALC is simply voice AGC moved to
the power amplifier chain with the difference that it's applied after
any spectrum filtering rather than before. So any sharp edges produced
appear as splatter.

{^_^} Joanne/W6MKU

[Steve]

Here's another one from the SDR# in C# (nasty language, but can be extracted to something useful):

https://www.assembla.com/code/sdrsharp/subversion/nodes/1166/trunk/Radio/AutomaticGain.cs

[Kristoff Bonne]

Hi Tony,

On 12-10-13 20:48, Tony Langdon wrote:

Another thing is that I have just noticed that the demodulated audio from rtl_fm has a terrible DC offset. My guess it is related to drift off the tuner in the dongle (which varies over time as these tuner are quite sensitive to temperature). I probably will need to get rid of that before any processing off the audio can be done.
Grr!!! Why do the details of implementing stuff aways make things so more difficult then first thought! :-(

A HPF would eliminate this as well as stray CTCSS tones (DC is 0 Hz afterall :) ).

I did a little bit on work on this a couple of days ago, and I seams to be getting a bit of a strange result.



This is a screencapture of audacity of the demoduted FM signal as produced by rtl_fm:



A clear DC offset. :-) (the values are floats going from -1 to +1)


OK, So I created a HPF with 40 taps around 300 Hz. Being lazy, I just used gnu octave:
f=300/8000
b=fir1(40,f,'high');

So, I would guess that the would have done the trick, but it does not. This is the result:



OK, the DC offset is much less, but it is still there. If we then apply an amplification of 5/1 on the audio-signal, we get this:




Better then the original audio but surely not perfect. :-(



73
kristoff - ON1ARF

[Alexandru Csete]

Kristoff,

You can eliminate the DC offset by subtracting the average value of all samples.

If you have a fixed number of samples with fixed offset you just calculate the average and subtract it.

If you have a continuous data steam where the offset may vary in time you must calculate the running average. There are many ways to do it, one way is to use a single pole IIR filter:

y[i] = alpha * x[i] + (1-alpha) * y[i-1]

where the parameter alpha will determine how fast you get to the true average.

This method is very efficient and is used in many if not all SDR receivers, since DC offset is a common "feature" of quadrature demodulators.

As for automatic gain control, I found that the AGC functions in GNU Radio didn't work well for analog voice application. I ended up using the AGC algorithm from Cutesdr by Moe Wheatley, which you can find described in great details in the Cutesdr technical manual available here:
http://sourceforge.net/projects/cutesdr/files/doc/

I hope this helps.

Alex

[Bruce Perens]

Why not just average the signal over a short period, which will give you the DC offset since the rest is close to symmetrical, and then subtract that value? Filters seem to be a bit of overkill.

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[Kristoff Bonne]

Hi Alexandru,

On 24-10-13 15:42, Alexandru Csete wrote:

Better then the original audio but surely not perfect. :-(

Kristoff,

You can eliminate the DC offset by subtracting the average value of all samples.
If you have a fixed number of samples with fixed offset you just calculate the average and subtract it.

That's also what I did in my first version, I averaged every out over a rolling-window of 1/2 second.

If you have a continuous data steam where the offset may vary in time you must calculate the running average. There are many ways to do it, one way is to use a single pole IIR filter:
y[i] = alpha * x[i] + (1-alpha) * y[i-1]

Well, I did a quick test adding this after the HPF and I get this:




I do have to drive up the amplification factor (this is +10 dB compaired to +7 dB in the earlier tests), but it does work pretty good.


Thanks!

where the parameter alpha will determine how fast you get to the true average.

The test above was simply with alpha is .5 and it works good.

The advantage of this value is that you can do it in integer math.

This method is very efficient and is used in many if not all SDR receivers, since DC offset is a common "feature" of quadrature demodulators.

Well, what is strange is that "rtl_fm" has an (experimental) option -C "DC blocking of output", but it does not seams to work very well. It produces a annoying high-piched noise. I don't know what exactly they do. (OK, I could check the source-code)

As for automatic gain control, I found that the AGC functions in GNU Radio didn't work well for analog voice application. I ended up using the AGC algorithm from Cutesdr by Moe Wheatley, which you can find described in great details in the Cutesdr technical manual available here:
http://sourceforge.net/projects/cutesdr/files/doc/

Now THAT is well written documentation! :-)

But it looks like the AGC is placed in a completely different place of the chain. In cutesdr, it's placed ahead of the demodulation. In my case, I deal with the demodulated audio.


Anycase, for this application, I can probably stick with a fixed amplication factor as it works good for what I need to do.

The goal is to have a stream of the signal that is transmitted by our clubstation -at a good audio-level- so it can be mixed in "thelinkbox" with the stream we receive from the remote stations and put both together on echolink. As the transmitted signal has know parameters that do not change, the AGC is probably not really needed here.

I hope this helps.

It sure did.

Alex

73
kristoff - ON1ARF

[Kristoff Bonne]

Hi Bruce,

On 24-10-13 02:54, Bruce Perens wrote:

Why not just average the signal over a short period, which will give you the DC offset since the rest is close to symmetrical, and then subtract that value? Filters seem to be a bit of overkill.

Well, I wanted to filter anyway to remove any effect of CTCSS subtones on the AGC, so -as Tony proposed- why not do both at the same time.


However, it turns out that the CTCSS signal are to high in amplitude to be removed by a simple 41 coefs filter. :-(

Anycase, it does help to reduce the DC component and the simple IIR filter as proposed by Alexandru is enough to get rid of that.




73
kristoff - ON1ARF