Fletcher-Munson compensation in BrutefirDrc?

[Gandhi]

Using the excellent LMS plugin BrutefirDrc I think it might be possible to include a new feature: Fletcher-Munson compensation. I believe compensating for the non-linearity of the human ear at different loudness levels would make listening to music even more enjoyable.

When using the digital volume control in the Squeezebox player, the plugin would have to notice volume changes and change the frequency response accordingly. (I'm not sure if this can be done on-the-fly, perhaps one has to restart the track?)

Naturally, one must also once calibrate the output from the Squeezebox to the actual SPL measured from the speakers, with the amplifier volume control fixed. This could be done using a file containing pink noise and a calibrated SPL meter.

This behaviour could instead of course be implemented in a separate LMS plugin, but I think it might be easier within the BrutefirDrc framework, as it (including the utility program BruteFIR) probably already includes all necessary DSP elements.

There is an ISO standard (https://en.wikipedia.org/wiki/ISO_226), which defines the equal-loudness contour.

Anyone up for the challenge? ;-)

Best Regards,
  Gandhi

[Chris Rudmin]

I tried this by routing the audio through SoX which has the loudness function you describe (http://sox.sourceforge.net/sox.html)

Never got it to work, but I didn't really try to debug this at all.

[Chris Rudmin]

If you already have brutefir working, you could just try adding the loudness command when the audio is piped to SoX.  In theory it should work. I never did get brutefir working.

[Gandhi]

Thanks Chris, a very interesting idea! I'll see if I have better luck! 

Best Regards, 
  Gandhi

[Olav Sunde]

An interesting idea Gandhi. I guess this would involve tweaking brutefirwrapper so we'd have to wait to see if Toby can look at it. In the mean time I'll send Chris the plugin so he can try it out.

Olav

[Olav Sunde]

Hi Chris and welcome

I'm sending you the plugin by PM so you can try a working version (provided you use LMS that is..)

Olav

[Klaas Reineke]

Sounds interesting do you have any idea how it should be applied? Take for granted that we would know the correct volume in phone. How would we translate this into the right loudness with sox?

Regards Klaas

[Gandhi]

According to Chris' brilliant suggestion, SoX could probably be used for this, thus making the solution much simpler than my initital proposal.

Excerpt from the SoX manual (http://sox.sourceforge.net/sox.html):

loudness [gain [reference]]
Loudness control - similar to the gain effect, but provides equalisation for the human
auditory system. See http://en.wikipedia.org/wiki/Loudness for a detailed description of
loudness. The gain is adjusted by the given gain parameter (usually negative) and the
signal equalised according to ISO 226 w.r.t. a reference level of 65dB, though an alternative reference level may be given if the original audio has been equalised for some other optimal level. A default gain of −10dB is used if a gain value is not given.

Possible solution

Using Sox, I think the problem could now be solved with the following steps:

• In beforehand:
• Calibrate the system using pink noise with the amplifier volume control for ever fixed, thus acquiring a reference level in dB.
• Adding a call to SoX in the template files, using the reference value as the second SoX parameter.
• On-the-fly:
  
• Notice volume level change and convert it to a negative dB value (the difference between the maximum level and the new level).
• Patch the call to SoX in custom-convert.conf, using the new value as the first SoX parameter. 
• Re-apply custom-convert.conf (hopefully, this could be done on-the-fly).
  

(Please note that I haven't had time to test SoX with the loudness effect yet, so my understanding of the parameters may not be correct.)

This proposal leaves all the gory DSP stuff to SoX, which - as I understand it - means that no changes to brutefirwrapper would be necessary.  

Best Regards,
  Gandhi

[Gandhi]

With "SoX parameter" above, I of course mean "SoX loudness parameter".

Best Regards,
  Gandhi

[Klaas Reineke]

the problem is that custom-convert.conf is normally only loaded at startup. I included a "hack" to reload it on change. I am not sure, what would happen if the file is changed freqently and the conversion are reloaded a lot of times. Sadly the current volume level is not available within the convert.conf environment via a variable.

Right now I do not have the time to fidle with this. One idea comes to mind: Just have a few loudness factors and select them with the remote control... Maybe better than nothing but still a few hours of coding that I cannot spare right now.

Regards Klaas

[Gandhi]

I completely understand your situation regarding plugin development versus home duties. I'm very happy that there now is a new official release!

Having a few equal loudness contours to choose from would go a long way in my opinion. For instance like the following modes, each with a user configurable dB value:

• Late night listening mode
• Soft listening mode
• Normal listening mode
• Critical listening mode
• Dance party mode.
  

Deviating from the exact equal loudness contour by manually changing the volume a modest bit, when in one of the above modes, is probably a totally acceptable approximation.  

If one could capture volume change events, another way of invoking the SoX loudness effect would be from within the brutefirwrapper program. (Or one could perhaps check the current volume value, say every time a track is started.)

As you probably understand, I'm not used to any of the programming languages used here, so I'm not sure exactly what the limits are, or where they are. Yet.

I'll try fiddling around a bit for myself, to satisfy my uncurbable curiosity.

Best Regards,
  Gandhi

[Klaas Reineke]

I played a little bit with the loudness setting. I got a problem: if you use something like -20dB for the loudness effect in sox the complete music gets quieter. This prevents me from using the digital out to decrease the volume because the decrease from the output and sox loudness is adding up. This is a complex effect that I cannot foretell so what to do?

I did not find any got tutorial how to use sox loudness on the web. I would need some ideas what the min and max ranges for the effect could be. The other question would be how to measure dB SPL as reference for all this but this is just the second step.

[Olav Sunde]

For loudness correction to work correctly you'd have to know a reference SPL for a given system and use this with sox somehow. This would also dictate a relation between the system volume control and sox/brutefirwrapper. In my setting this would not work as I use the volume control of my Devialet amp. Also, as Klaas says, you'd need attenuation with loudness to avoid digital clipping. The filter I use with brutefir is already attenuated quite a bit (because I do lots of correction in the bass region). Additional attenuation is not good in my setting.

There are other issues with current BrutefirDRC that may be more important to focus on than the loudness correction. Performance through BrutefirDRC is rather sluggish at high sample rates. I hear no dropouts of sound, but CPU load is higher than I expect and gapless is not working well. Down sampling  does not work (i.e. material higher than 48kHz sent to an SB2/3 will not be handled correctly by LMS with BrutefirDRC enabled). Noise, or 'static' as some calls it, is mixed with music on track transitions for 16/44.1kHz material when you send PCM to the squeezebox.

Olav

[Gandhi]

Thank you for fiddling, Klaas!

(I see that Olav already has answered some of the questions, but here comes the reply I already had prepared, as is.)

I have also been looking for information regarding how SoX Loudness, without success.  Perhaps SoX automatically lowers the volume to avoid clipping, as the bass gets quit a large boost in the lower SPL range? Maybe it would still work in an interval, if one also attenuates the level a bit in beforehand with SoX? I'll see if I can find an explanation in the SoX source code.

I have been playing a bit myself, as I thought you might be busy. I'm trying to change brutefirwrapper to get the current volume value and use SoX accordingly. However, as I have never done any work in Python before, my progress is rather slow and painful. Right now, I'm able to get the current Squeezebox volume with a Python script.

As for acquiring a reference level, the following procedure is what I'm thinking:

• Use Replay Gain to normalise all music to 89 dB:
• This is according to the 89 dB Fletcher Munson contour for pink noise in stereo, as I understand it
• The same 89 dB is the default in dBpowerAMP and foobar2000
• Perhaps one would always have to use Track Gain and never Album Gain?
• Calibration:
• Set the volume for Squeezebox to 0 dBFS (maximum volume)
• Use no EQ, attenuation or other sound processing anywhere in the stereo system
• Use Squeezebox to play a music file containing pink noise at 0 dBFS in stereo
• Vary the volume control for the amplifier until 89 dB(C) is achieved in the listening position, using a calibrated SPL meter.
• The volume control for the amplifier is then held fix for ever.
• Adjustment while listening:
• Loudness effect in SoX:
• Set the second parameter to the reference level 89 dB.
• Set the first  parameter according to a chosen contour, or to the dB value calculated from the current volume value in the Squeezebox.
• I would instead play pink noise at -14 dBFS and use a reference level of 75 dB, to be nice to both my speakers and my ears.
  

Best Regards,
  Gandhi

[Gandhi]

Oh, and perhaps I'd better add that this is an idea that I thought would be interesting to try. Maybe efter experimenting it'll turn out that the disadvantages outweigh the advantages.

I naturally wouldn't want to waste other peoples time. If there are more pressing issues to attend to, especially the kind of problem Olav mentions above, then by all means concentrate on those instead.

But if nothing else, I'll keep on tinkering with this for myself. It's fun to experiment with audio. It's also fun to learn a new programming language and it helps a lot to have a reasonably difficult goal to work towards. When I'm done, I'll of course share the code with any interested persons.

Best Regards,
  Gandhi

[Klaas Reineke]

Hi Olav,

I got an idea for your samplerate problem: just play flac, then the material with higer sampling rates is handled fine. I cound never distinguish between flac and pcm anyway. I can play my high res material fine via flac.

Could you explain the problems with high sampling rates in more detail, maybe in a different thread? Alternativly if already posted provide the links :)

Best regards Klaas

[Klaas Reineke]

Hi Gandhi,

I experiment myself with this. The sox parameter is easy to understand, I just had to read the manpage. It is like gain, they really mean it. So it attenuates the audio by the given dB range according to the Iso 226 curves based on a reference level of 65dB. Reference can only be changed between 25 and 75dB so you can never got to the 86 or whatever with sox.

My experiments still did improve the sound with low volume a lot! Because my wife is not really into loud music I will experiment further with loudness. I will do it completely server based, because I have no Python experience and I do not want to tinker with the script. Because of this I cannot fix the other issues that are script related...

If a make some progress I will post my results here...

Regards Klaas

[Olav Sunde]

Hi Klaas,

I'll answer here as I do not think a new thread is needed at the moment.
When Toby enabled 88.2 and 96kHz handling in brutefirwrapper it was first sluggish, then very nice after he had worked with it for some time. 176.4 and 192kHz playback was until recently only possible through software with Vortexboxplayer. After triode made the EDO kernel fix for Touch this spring, 176.4 and 192kHz was available on a hardware player. Recently triode has also written Squeezelite http://forums.slimdevices.com/showthread.php?97046 , a linux headless player that handles all up to 192kHz.

Toby used an SB2 or 3 so he actually never played the higher sample rates native. 176.4 and 192kHz wasn't even an option when he last reworked brutefirwrapper. My point is that Toby at some point needs to optimize brutefirwrapper for all sample rates. Earlier he'd have to buy a Touch to do this. Now he can do this testing with Squeezelite. I believe an optimisation will also uncover the bug that creates the white noise on track transitions for 16/44.1 material.


I know that flac and mp3 is working fine in all modes with the version that is posted now. As for the difference between streaming flac and PCM--as long as I stream through a Touch for playback, PCM decoding on the server is clearly better in every respect. So sending flac to Touch for decoding is not an option for me. If I later set up triode's Squeezelite (or a Vortexbox) on a linux box close to my amp I'd expect the advantage of server side decoding of flac to be neglible.
Many audiophiles will stream PCM to their squeezeboxes. That is why the option to select PCM is there and I believe this should work as smooth as flac and mp3.

Best regards Olav

[Gandhi]

@Klaas
Yes, the maximum reference value is 75 dB. As I understand it, simply changing 89dB to 75dB in my calibration procedure above would make it work. (The reference contour used in Replay Gain is an average contour anyway.)

I usually prefer using my amplifier volume control, as I believe it's better implemented than the digital one in the Squeezebox. However, as the sound quality at lower listening levels generally is not satisfactory, for me it's completely justifiable to use the one in the Squeezebox instead, if the Fletcher-Munson compensation performs well enough. I've only
done a little experimenting with different Fletcher-Munson contours and I think I need to do a lot more. (I haven't yet found the energy to calibrate the system according to my own proposal...) But so far I think that it's promising, and that it might proove to be a good idea, at least in a lower interval. But it might not be such a good idea for critical listening at higher SPL's, if it doesn't perform well. Therefore, an off switch (or perhaps a threshold level, over which F-M is not used) could be useful, if implemented.

I'm glad that you have noticed that this also has a substantial WAF (or SAF, to be PC) and it also has a lower WCUF, for those of you with small ones, which hopefully increases the SAF even further.

Keep up the good work! (And I'll keep on struggling.)

Best Regards,
  Gandhi

[Klaas Reineke]

Hi Olav,

thanks for the information. I am right now considering to switch to a Softwareplayer using an HiFace 2 that can stream 192kHz. I thought to use a patched Squeezeplay see http://www.mail-archive.com/un...@lists.slimdevices.com/msg20025.html for this. Maybe I need to checkout the Squeezelite...

Right now I am just using a Squeezbox 3 so everything is resampled to 44.1 or 48kHz. I have some HiRes material so I really would like to stream it without downsampling... Hopefully Toby finds the time to play with the wrapper script :)

Regards Klaas

[Mervin Beng]

Hi Olav,

I have gapless working perfectly on my setup. I recall that some time ago Toby and I spent some time tracking down gapless issues to the point that my classical music which has tracks separating continuous music work with no audible artefacts (using SB Touch).

I agree - brutefir attenuation is something we can't run away from, even if boost is kept down. I have to use ~10dB attenuation to avoid clipping. Any additional attenuation for loudness compensation would not be acceptable for my usage.

What are the performance characteristics you see? I can process two independent 24/96 streams to two SBTs without hitting 40% CPU usage on the server. The filters are at 44.1 or 48kHz. Any higher is moot for me, as the measurement chain is limited to 48kHz. I know I don't hear much beyond 12-15kHz... age is my enemy... and sample width is the priority for me really. IIRC, if you reconvert back to flac after brutefir, it does take up more CPU. I stream PCM to the SBT to save CPU, at the expense of network bandwidth. However even with wifi for one of the SBT, there is no dropout at 96kHz.

Regards,
Mervin

[Klaas Reineke]

Hi Gandhi,

I will continue to experiment with this. I used a track of audiophile music and sox loudness set to -50dB with reference of 75dB. It was a stunning experience. You have nearly no volume at all it was quiet, but it still had a lot of bass and treble.... I would not use it for serious loud listening, but I am going to dig deeper for normal and low volume listing. I would not be to precise about the reference stuff, because it was an eye opening experience without any SPL reference volume measurement etc...

Regards Klaas

[Olav Sunde]

Hi Mervin

I do not quite understand where the noise on track transitions originates. It only occurs with 16/44.1 material. If I convert files to 24/44.1 the problem is gone. I have tested filter sample rates of 44.1, 48 and 96kHz with brutefir. Same result. Is your brutefirwrapper different from the one in the release? Any tweaks in there that could be used in the release version?

My Readynas has a Atom 1.8 dual core. With a 48kHz filter I see 6-8% load per core for 44.1 and 48kHz material, around 30% per core for 24/192. On a single 1.6 cpu laptop with Vortexbox load is 13-15% for 44.1 and 48kHz. I never experience dropouts.

I create filters with Audiolense 4.4XO. AL applies attenuation in the filter so I do not have to check for clipping in brutefir, but without this I would probably use 20dB of attenuation.

Best regards  Olav

[Mervin Beng]

As far as I can see, there is little optimisation to be done for higher sample rates. A very large percentage of CPU is spent in sox/brutefir/flac pipe, so hardly anything in Python can change the CPU usage.

That said,

1. Stream PCM to SBT. It will save one flac process per channel.
2. Use a 64-bit version of brutefir, compiled with a recent compiler. The 64-bit fft libraries should help.
3. Instead of the 32-bit sox that is packaged with LMS, use the 64-bit sox on your distro. Do this by renaming the 32-bit binaries in the /opt directory.

The pipeline that processes a 16/44.1 and 24/44.1 is identical. The only difference that I can think of is that sox converts 16 bits to 24 bits as needed by brutefir. In the wrapper there is no difference whatsoever, so you might want to simulate the pipeline on your server (remember to use the LMS binaries, not the ones supplied on the distro), and check if the resulting output is identical.

A quick stab at the gapless problem. I don't have time to test for at least some weeks, so please give it a try:

1. The reverb tail is written at the end of each track:

tail_bytes = int(samplerate*TAIL_SECONDS)*CHANNELS*format.width

2. The tail file is pcm. IIRC, the tail file is *always* 24 bit. At least on my filter it says:

output "l_out","r_out" {
device: "file" {path: "/dev/stdout";};  # module and parameters to put audio
sample: "S24_LE"; # sample format

3. If this is the case, the format.width may be the cause of the problem. I believe for the tail file, format.width should be replaced with brutefir_out_width, which is 3. The section which joins the tail file to the start of the current track will also need to be corrected.

Let me know if this works!

Cheers,
Mervin

[Olav Sunde]

This is actually outside my comfort zone..  Yes, S24_LE is the format used in filters
I see nine lines i brutefirwrapper with the format.width parameter:

    Line 289:         chunk = our_stdin.read(int(BLOCK_SECONDS*samplerate)*CHANNELS*format.width)
    Line 327:                     samplerate * format.width * 2,
    Line 328:                     format.width * 2,
    Line 343:             data = brutefir_stdout.read(format.width)
    Line 344:             if len(data)!=format.width:
    Line 356:     readahead_bytes = readahead_samples*format.width
    Line 370:         chunk = brutefir_stdout.read(int(BLOCK_SECONDS*samplerate)*CHANNELS*format.width)
    Line 378:     tail_bytes = int(samplerate*TAIL_SECONDS)*CHANNELS*format.width
    Line 467:         return [ format.decode(body[offset:offset+format.width]) for offset in range(0,len(body),format.width) ]
 
What should be changed to achieve your suggestion? Sorry for being  a bit dense here.

Olav

[Mervin Beng]

I believe that this is the answer. It's likely to be a bug on brutefirwrapper. I'll try to send you a replacement brutefirwrapper to test. It may be not be based on the very latest version, but is pretty up to date. It might take a day of two. It's easier for you to test, than for me to create 16/44.1 test tracks that have no gaps between tracks.

Mervin

[Olav Sunde]

Good. I'll do some testing when I get the file.

Olav

[Mervin Beng]

Sorry Olav, I've looked at the code again, and my guess is incorrect, as the wrapper seems to check the output format correctly, so the tail which is used for the gapless playback should be correct. Don't hold your breath for any fix from me. I'll keep looking at it in my spare time.

Regards,
Mervin

[Olav Sunde]

That is OK Mervin. We'll find out of this at some point.

Best regards Olav

[Klaas Reineke]

Hi there,

today I realeased a new version 3.0-alpha on Sourceforge: https://sourceforge.net/projects/brutefirdrc/files/latest/download

It contains my first stab on loudness correction using sox build in loudness function. It works pretty amazing in the current state. I would appreciate some feedback from all of you.

To install it:

1. enable loudness in the Plugin global settings
2. enable loudness in the BrutefirDrc menu for the player (player local configuration)
3. restart the server (I use the Perl Replace Module to dynamically replace some basic LMS functionality, this did only reliable work on startup for me)

To disable:

Just disable the global switch, then the replacements for LMS are not loaded on next start of the server.

How it works (bear with my english):

1. I replaced the way the volume switch is working.
2. it reads out the attenuation already done by the loudness feature and decreases volume by the missing dB
3. the loudness feature from sox is used to decrease the volume by the dB selected via the volume switch
1. the loudness setting in sox can only be set on at the beginning of a track
2. to make it possible to increase the volume I am using a factor of 2/3 of the attenuation as gain afterwards, i.e. attenuation 30 dB => gain 20 dB => real attenuation 10 dB via sox
3. the missing attenuation is added via the normal volume switch of the squeezebox, i.e. 20 dB. By this it is possible to increase the volume by 20 dB for the currently playing song. More is not possible because the squeezebox is at max volume and the attenuation via sox cannot be changed on the fly
4. for each newSong the volume is resetted and all values are recalculated.
5. it is recommended to start a new track if the volume is changed by big amounts

Tested and working on:

• Squeezebox Classic
  
• Squeezeeleite software client

It should work on any device that is using the Squeezebox2.pm file within the LMS server. If it is not working for you, just post me a message with the device you are using.

@Gandhi: I guess it is pretty much what you wanted.

@Mervin: you should try it out. I did take the filter based attenuation into account. Meaning that I read out the attenuation from the filter and replace it with attenuation of 0dB. If you use the volume control the attenuation for the filter is first done via the sox loudness effect. If not enough attenuation is there, i.e. only 5dB from volume control but 10dB needed in filter, the plugin will attenuate with an additional 5dB. So the minimal attenuation is based on the filter settings. No attenuation is added up.

Just ask questions and provide me with plenty of feedback. Right now I am cleaning up the sourcecode and try to move the filter setting to the player specific configuration.

Regards Klaas

[Klaas Reineke]

I forgot one thing:

I did not encounter clipping so far. My factor of attenuation in db * 2/3 => gain, may lead to clipping. If you encounter clipping please send me what have been your volume settings, what attenuation the filter needed, etc.

Regards Klaas