Brutefir Filters not working

[robert....@btinternet.com]

Not sure if theis group is still being read but...

I've been using Brutefir on an Odroid-XU4 sucessfully for a couple of years but only for the equalizer and it works well.

I recently obtaimed a UMIK-1 calibrated USB microphone so I could try doing room correction using REW. I've carried out left and
right tone sweeps and created filters using REW but REW creates .wav files and not .pcm although the filters are .pcm signed.
I tried converting .wav -> .pcm without much luck but eventually found that stripping off the first 44 bytes from the .wav converted
it into a .pcm. Suffice to say, I tried the .wav files natively, tried renaming to .pcm and then tried converting the .wav -> .pcm
and Brutefir produced no output with any of those files. so I went back to first principles and tried using the filters provided
with the example files. That didn't work either! There must be a fundamental problem somewhere!

Here is the config file I used - basically the one provided in the example files with only paths changed...

------------------------------------------------------------------------------------------------------------------------------------

## DEFAULT GENERAL SETTINGS ##

float_bits: 32; # internal floating point precision
sampling_rate: 48000; # sampling rate in Hz of audio interfaces. Must match sample rate of filter .pcm if present in config
filter_length: 16384,4; # length of filters
overflow_warnings: true; # echo warnings to stderr if overflow occurs
show_progress: false; # echo filtering progress to stderr
max_dither_table_size: 0; # maximum size in bytes of precalculated dither
allow_poll_mode: false; # allow use of input poll mode
modules_path: "/usr/lib/brutefir"; # extra path where to find BruteFIR modules
monitor_rate: false; # monitor sample rate
powersave: true; # pause filtering when input is zero
lock_memory: false; # try to lock memory if realtime prio is set
convolver_config: "/usr/share/squeezeboxserver/BrutefirDrc/wisdom/.brutefir_convolver"; # location of convolver config file

coeff "drc_l" {
filename: "/usr/share/squeezeboxserver/BrutefirDrc/filters/null-65k48-l.pcm";
format: "FLOAT_LE"; # file format
attenuation: 0.0; # attenuation in dB
blocks: -1; # how long in blocks
skip: 0; # how many bytes to skip
shared_mem: false; # allocate in shared memory
};

coeff "drc_r" {
filename: "/usr/share/squeezeboxserver/BrutefirDrc/filters/null-65k48-r.pcm";
format: "FLOAT_LE"; # file format
attenuation: 0.0; # attenuation in dB
blocks: -1; # how long in blocks
skip: 0; # how many bytes to skip
shared_mem: false; # allocate in shared memory
};

## INPUT DEFAULTS ##

input "l_in","r_in" {
device: "file" {path: "/dev/stdin";}; # module and parameters to get audio
sample: "S24_LE"; # sample format
channels: 2/0,1; # number of open channels / which to use
delay: 0,0; # delay in samples for each channel
maxdelay: -1; # max delay for variable delays
mute: false,false; # mute active on startup for each channel
};

## OUTPUT DEFAULTS ##

output "l_out","r_out" {
device: "file" {path: "/dev/stdout";}; # module and parameters to put audio
sample: "S24_LE"; # sample format
channels: 2/0,1; # number of open channels / which to use
delay: 0,0; # delay in samples for each channel
maxdelay: -1; # max delay for variable delays
mute: false,false; # mute active on startup for each channel
dither: false; # apply dither
};

# Digital room correction
filter "l_drc" {
from_inputs: "l_in"/0.00;
to_outputs: "l_out";
process: -1; # process index to run in (-1 means auto)
coeff: "drc_l";
delay: 0; # predelay, in blocks
crossfade: false; # crossfade when coefficient is changed
};

filter "r_drc" {
from_inputs: "r_in"/0.00;
to_outputs: "r_out";
process: -1; # process index to run in (-1 means auto)
coeff: "drc_r";
delay: 0; # predelay, in blocks
crossfade: false; # crossfade when coefficient is changed
};

# filter_attenuation=0.00

------------------------------------------------------------------------------------------------------------------------------------

And here is the relavent output from the log file...

======== brutefirwrapper rev 12 starting at 1497507338 2017-06-15 06:15:38
Resampling brutefir filter from 48000 to 44100
Patching brutefir input format from S24_LE to S16_LE
Patching brutefir output format from S24_LE to S16_LE
Traceback (most recent call last):
touch: cannot touch '/usr/share/squeezeboxserver/.brutefir_config': Permission denied
File "/usr/share/squeezeboxserver/Plugins/BrutefirDrc/Bin/brutefirwrapper", line 466, in read_tail
age = time.time()-os.stat(tail_filename).st_mtime
OSError: [Errno 2] No such file or directory: '/tmp/.BrutefirDrc-123/tail-00_1e_06_31_09_e8.pcm'
Output follows (1.775s)

BruteFIR v1.0m (November 2013) (c) Anders Torger

Internal resolution is 32 bit floating point.
Creating 4 FFTW plans of size 32768...finished.
Loading 2 coefficient sets...Memory allocation failure (0 bytes), terminating program.
Output complete (1.798s)

------------------------------------------------------------------------------------------------------------------------------------

In the log file the only relavent information, I think, is...

Loading 2 coefficient sets...Memory allocation failure (0 bytes), terminating program.
Output complete (1.798s)

That error was present using the files I converted from .wav -> .pcm as well. Bearing in mind that the equalizer alone works I'm fairly sure Brutefir is working properly so I'm guessing something is wrong with the config
file. I've tried with filters+eq and filters alone as shown above.

Any help or suggestions or advice appreciated.

Bob

[Olav Sunde]

Hi
the first you should do is to fix permissions that you see in the log file. Make sure squeezeboxsever can write to folders. Next check /tmp. Is there a .BrutefirDrc-* folder? Check file pernissions here.
If you use the default demo config, make sure the pcm filter files are in the right (expected) locations. Your config file looks OK, so there must something else going on here.

I use sox to prepare a stereo wav file as single channel PCM.

sox "stereo.wav" -e floating-point -t raw filter-l.pcm remix 1
sox "stereo.wav" -e floating-point -t raw filter-r.pcm remix 2

[robert....@btinternet.com]

Hi Olav,
I'm using the Logitech Media Server plugin by Klaas Reinke (? Spelling) on Ubuntu 16.04 LTS so file paths are probably different to the usual paths. All the required filters appear to be in the correct places - the Brutefir Equalizer runs without any problems. It's only when I try to add in room correction filters that it doesn't work. I've fixed permission problems with /tmp but still get the 'tail' error. Would it be possible for you to send me a copy of your working left and right filters so I can give them a try? I know they won't be relavent to my room but at least I can see if known working filters will work in my setup.

Bob

[Olav Sunde]

Hi
I do not think the 'tail' error is relevant to your problem. This turns up when brutefirwrapper looks for a tail to mix in before the next track. If one selects a different track (from a different album for example) we will see this message, but brutefirwrapper ignores it and continues.
I am not using LMS at the moment, but have a working installation so I will try to do some testing during the weekend and get back to you.
Is sox installed and working? Is there a sox binary as part of LMS? I think brutefirwrapper uses a hardcoded path for sox.

Olav

[robert....@btinternet.com]

Hi
Thank you for your help.
brutefirwrapper is coded for a specific sox path but I edited the path a couple of years ago so that it points to the sox binary already provided by LMS. I also installed the sox binary in the Linux 'path' as well just in case...

Bob

[Olav Sunde]

I have sent you a PM

Olav

[robert....@btinternet.com]

Hi Olav,
I've tried the files you sent and they are still not working. I'm getting the same error about memory allocation and then program termination :(
Maybe there is an issue with Brutefir memory usage but I have no idea how to manipulate that in the config file. According to the Brutefir
documentation there may be ways to modify memory allocation but it's all a bit beyond my understanding.

[Olav Sunde]

Did you use only the files I sent or did you include your equaliser settings? I have seen the memory allocation error, but only if I tried to use convolution and (brutefir)eq at the same time. Still. convolution on it's own worked. This was on a Intel i5 nuc with debian. I am still wondering why this happens. What happens if you log in as root? Does brutefir run when started from there?
What does python -V say?
In desperation I have some times used chown -R 777 on folders to make things work...
Does brutefirdrc.prefs look OK? Normally located under ......squeezeboxserver/prefs/plugin

Maybe if you create soft links in /etc to where you have placed BrutefirDrc? I.e. /etc/squeezeboxserver/BrutefirDrc/filters and settings..

Olav

[robert....@btinternet.com]

I only used the files you supplied - no attempt to include Eq. Python version = 2.7.12 and logging in as root made no difference at all. Created soft links and tried chmod -R 777 and nothing works. Even tried modifying memory references in config and still no luck :(

[Olav Sunde]

I have no experience with ARM CPUs and brutefir, but it is strange that brutefir will run with the EQ config and not convolution. The EQ part is of course internal to brutefir while convolution uses external files, but brutefir still allocates memory in the same manner. The only option left as I see it (apart from re-installing the plugin with the version I run) is using shorter filters. I.e. 32k, 16k or 8k taps in stead of the 65k sample.

I run a tail -f -n 25 on brutefir.log What does it say when you use the EQ?

Olav

[robert....@btinternet.com]

I tried the 16k files you sent but still get the same error 'memory allocation failure'
The following is output from log file, using EQ only, for a playing track...

======== brutefirwrapper rev 12 starting at 1497785525 2017-06-18 11:32:05
Resampling brutefir filter from 192000 to 44100
ignoring tail; it is 54s old
Output follows (0.443s)

BruteFIR v1.0m (November 2013) (c) Anders Torger

Internal resolution is 32 bit floating point.

Creating 4 FFTW plans of size 8192...finished.
Loading coefficient set...finished.
Warning: internal resolution not high enough to dither (outputs 0 - 1).
Warning: no support for clock cycle counter on this platform.
Timers for benchmarking may be unreliable.
Filters in process 0: 0
Filters in process 1: 1
Creating inverse inplace FFTW plan of size 16384 using wisdom...finished
Audio processing starts now
Readahead complete (0.774s)
peak: 0/0/-0.30 1/30/+0.93
peak: 0/0/-0.22 1/35/+0.93
peak: 0/0/-0.22 1/44/+0.93
peak: 0/193/+2.40 1/122/+1.88
peak: 0/304/+2.40 1/186/+1.88
peak: 0/304/+2.40 1/190/+1.88
peak: 0/317/+2.40 1/199/+1.88
peak: 0/416/+2.40 1/233/+1.88
peak: 0/485/+2.40 1/316/+2.15
peak: 0/489/+2.40 1/320/+2.15
peak: 0/498/+2.40 1/320/+2.15
peak: 0/498/+2.40 1/330/+2.15
peak: 0/498/+2.40 1/334/+2.15
peak: 0/559/+2.40 1/402/+2.16
peak: 0/715/+2.40 1/549/+2.16
peak: 0/750/+2.40 1/588/+2.16
peak: 0/758/+2.40 1/638/+2.16
peak: 0/759/+2.40 1/638/+2.16
peak: 0/779/+2.40 1/644/+2.16
peak: 0/779/+2.40 1/647/+2.16
peak: 0/806/+2.40 1/704/+3.16

Finished!
Output complete (209.603s)

[Olav Sunde]

In the log file I notice that it says 8192. I've sent the same filter with 8192 taps in a PM.

More worrying is the warning "...no support for clock cycle..." Maybe brutefir needs this for convolution?

There is also a lot of clipping occuring. You should add 3.2 dB attenuation in the config file to avoid this:

from_inputs: "x_in"/3.2;

Olav

[robert....@btinternet.com]

Tried the 8k files you sent and still no Audio :(
Im not very conversant with the config files. Do you mean
l_in/3.2
r_in/3.2

and where would I put them in the following (my standard EQ config)?

## DEFAULT GENERAL SETTINGS ##

float_bits: 32; # internal floating point precision

sampling_rate: 192000; # sampling rate in Hz of audio interfaces

filter_length: 16384,4; # length of filters
overflow_warnings: true; # echo warnings to stderr if overflow occurs
show_progress: false; # echo filtering progress to stderr
max_dither_table_size: 0; # maximum size in bytes of precalculated dither
allow_poll_mode: false; # allow use of input poll mode
modules_path: "/usr/lib/brutefir"; # extra path where to find BruteFIR modules
monitor_rate: false; # monitor sample rate
powersave: true; # pause filtering when input is zero
lock_memory: false; # try to lock memory if realtime prio is set
convolver_config: "/usr/share/squeezeboxserver/BrutefirDrc/wisdom/.brutefir_convolver"; # location of convolver config file

logic:
"eq" {
{
coeff: "equalizer";
bands: 20, 40, 60, 100, 800, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000, 12000, 16000;
magnitude: 20/0.0, 40/5.0, 60/5.0, 100/5.0, 800/0.0, 1000/0.0, 2000/0.0, 3000/0.0, 4000/0.0, 5000/0.0, 6000/0.0, 7000/0.0, 8000/0.0, 9000/0.0, 10000/0.0, 12000/0.0, 16000/-3.0;
};
};

coeff "equalizer" {
filename: "dirac pulse";
shared_mem: true;
blocks: 4;
};

## INPUT DEFAULTS ##

input "l_in","r_in" {
device: "file" {path: "/dev/stdin";}; # module and parameters to get audio
sample: "S24_LE"; # sample format
channels: 2/0,1; # number of open channels / which to use
delay: 0,0; # delay in samples for each channel
maxdelay: -1; # max delay for variable delays
mute: false,false; # mute active on startup for each channel
};

## OUTPUT DEFAULTS ##

output "l_out","r_out" {
device: "file" {path: "/dev/stdout";}; # module and parameters to put audio
sample: "S24_LE"; # sample format
channels: 2/0,1; # number of open channels / which to use
delay: 0,0; # delay in samples for each channel
maxdelay: -1; # max delay for variable delays
mute: false,false; # mute active on startup for each channel

dither: true; # apply dither
};

# Add equalizer
filter "eq_l" {

from_inputs: "l_in"/0.00;
to_outputs: "l_out";
process: -1; # process index to run in (-1 means auto)

coeff: "equalizer";

delay: 0; # predelay, in blocks
crossfade: false; # crossfade when coefficient is changed
};

filter "eq_r" {

from_inputs: "r_in"/0.00;
to_outputs: "r_out";
process: -1; # process index to run in (-1 means auto)

coeff: "equalizer";

[robert....@btinternet.com]

Sorry, meant to ask where do I edit into my config file...
from_inputs:"x_in"/3.2

and is it just as above or should it be two entries...
from_inputs:"l_in"/3.2
from_inputs:"r_in"/3.2

[Olav Sunde]

Under this header at the end of the config file.

# Add equalizer

filter "eq_l" {

from_inputs: "l_in"/3.20;
......

from_inputs: "r_in"/3.20;

[robert....@btinternet.com]

Thank you and sorry, yes, that was obvious :)
It's very hot here today and I've been looking at too many files!
Where do I go from here?
I could try setting Brutefir up on Window 10 to see if that works but that goes away from the hardware solution I want.
I guess I could try entering correctins direct into EQ but I don't know if that would work too well. No paramertic equalization!

Bob

[Olav Sunde]

Unfortunately, brutefir and BrutefirDrc is unix/linux only so Win 10 would not work. A linux distro directly on your laptop (or in a virtual machine in W10) would probably do the trick. Also Roon will do both convolution and parametric EQ on your Win10 PC, but Roon is not free....  There is a two week free trial to get you started though - and Roon supports streaming to squeezeboxes (i.e. slimproto)
I have used LMS/BrutefirDrc until recently, waiting for Roon to include convolution. They now have a full DSP with upsampling, PEQ and more - and the sound quality is simply stellar!
The fact that BrutefirDrc is not developed any more comes as a very strong reason to look elsewhere.
I love LMS with iPeng. If BrutefirDrc was still maintained I'd probably use it longer, but...

Olav

[Olav Sunde]

I understand completely! It takes strong of concentration to solve errors like this. I have used lots of time with BrutefirDrc over the years

Did you compile brutefir yourself? Maybe you could install a pre-compiled version from a repo and hope that this would work?

Olav

On Sunday, 18 June 2017 18:50:50 UTC+2, robert....@btinternet.com wrote:

[robert....@btinternet.com]

I didn't compile brutefir myself, I used the binaries compiled by 'jackofall' for CSOS. I tried compiling it myself but ended up with all sorts of errors in brutefir.log so switched to the CSOS version. There is also a repo install of brutefir for ARM Ubuntu which I tried but that produces the same memory allocation error. I'm not at all conversant with compiling but from what 'jackofall' said some time ago on the squeezebox forums I would guess there are lots of compile flags that could be used to tune the way brutefir works. But that's way beyond my knowledge!

Bob

[Olav Sunde]

Then I fear it is an in-compatibility between your processor or memory controller and brutefir. I also use pre-compiled binaries as I find it too much to compile from source. I have always used x86 processors so no problems like this. I know that 'jackofall' included BrutefirDrc in his project, but I have no details

Olav

[robert....@btinternet.com]

I may try to get hold of a driveless i5 or i7 NUC plus a cheap SSD and run Ubuntu on it - would need USB 3 or eSATA and HDMI. They do seem expensive though!

Bob

[Olav Sunde]

I have used a NUC for some years now. An i5 DC53427HYE with HDMI + other necessary connections.This one has a fan so it makes some noise. I use an internal m-sata disk for boot. Music files on a NAS. At the moment I run Debian with Roon Core on top, but LMS and BrutefirDrc runs fine too. Be aware that the latest perl does not work well with parts of the config code for BrutefirDrc on LMS. It is relatively easy to work around though. Brutefir itself and the wrapper runs fine and is not affected.

Olav

[robert....@btinternet.com]

I've ordered an Intel NUC from Amazon with 8Gbyte RAM and 120Gbyte M.2 card. Should be here tomorrow. All I've got to work out now is how to get Ubuntu on board - probably a bootable memory stick.

Bob

[robert....@btinternet.com]

Hi Olav,

I've got my NUC set up now and have been trying brutefir EQ and Filters. EQ works fine but Filters still does not work.
Filter is using the examples only - sample_filter_no_eq.txt, null-65k48-l.pcm, null-65k48-r.pcm,with no changes other than paths.
Here is the log output for both with EQ first followed by filter...

======== brutefirwrapper rev 12 starting at 1498464742 2017-06-26 08:12:22
Resampling brutefir filter from 48000 to 44100
ignoring tail; it is 58s old
Output follows (2.066s)

BruteFIR v1.0m (November 2013) (c) Anders Torger

Internal resolution is 32 bit floating point.

SSE capability detected -- optimisation enabled.
Creating 4 FFTW plans of size 32768...finished.

Loading coefficient set...finished.
Warning: internal resolution not high enough to dither (outputs 0 - 1).

Estimated CPU clock rate is 3092.907 MHz. CPU count is 4.

Filters in process 0: 0
Filters in process 1: 1

Creating inverse inplace FFTW plan of size 65536 using wisdom...finished
Audio processing starts now
Readahead complete (3.002s)

Finished!
Output complete (119.831s)

======== brutefirwrapper rev 12 starting at 1498406520 2017-06-25 16:02:00

Resampling brutefir filter from 48000 to 44100

Traceback (most recent call last):

File "/usr/share/squeezeboxserver/Plugins/BrutefirDrc/Bin/brutefirwrapper", line 579, in <module>
main()
File "/usr/share/squeezeboxserver/Plugins/BrutefirDrc/Bin/brutefirwrapper", line 123, in main
filter_name = patch_filter(filter_name,client_id,samplerate,in_format,opts.copy_sample_format)
File "/usr/share/squeezeboxserver/Plugins/BrutefirDrc/Bin/brutefirwrapper", line 181, in patch_filter
filter_body = patch_samplerate(filter_body,samplerate)
File "/usr/share/squeezeboxserver/Plugins/BrutefirDrc/Bin/brutefirwrapper", line 231, in patch_samplerate
filter_body,junk = re.subn(r'\bfilename\s*:\s*"(.*)"\s*;',resample_filter,filter_body)
File "/usr/lib/python2.7/re.py", line 166, in subn
return _compile(pattern, flags).subn(repl, string, count)
File "/usr/share/squeezeboxserver/Plugins/BrutefirDrc/Bin/brutefirwrapper", line 225, in resample_filter
'rate','-v',str(samplerate),
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception

[Olav Sunde]

I believe this might have something to do with how you have installed the plugin (or maybe the plugin itself is broken...)

I have sent you a PM

On Friday, 16 June 2017 13:58:00 UTC+2, robert....@btinternet.com wrote:

[robert....@btinternet.com]

Hi Olav,

Thank you for the plugin and instructions for use. I tried them out as per instructions and couldn't get LMS to restart. All that was shown in the LMS logs was failure to start and continuous attempts to restart. I've reverted to my previous install to get LMS running again and will investigate by following your instructions one step at a time, restart LMS before going to the next step and look at the logs to see if they show anything of use. I'll try it out when I have lots of time so that I can take much more care. I'll get back to you when I've found the problem.

Bob