WHY BLIND A/B TESTING IS IRRELEVANT
Mkuller
it and comparing it to my old amp. In this case, the two amplifiers sound much
more similar than I would have expected, but I hear small differences between
them that I consider important, especially for long term listening pleasure.
Now when I sit and listen to music I relax and enjoy the experience. When I
decide it's time to switch them, I listen to 3 or 4 cuts that demonstrate
specific musical qualities and take notes. After careful level matching (with
a meter), I switch the amps and listen to the same music in the reverse order
and take notes again. As I said, I have consistantly noted important subtle
differences in comparing the sound of the two amps playing music. One of the
two amps would not satisfy me as much long term, particularly after having
listened at length to the other one.
However, I am absolutely convinced that if I were to listen to the amps blind,
I would not be able to identilfy which one I was listening to. The standard
"objectivist" line is that if you can't identify the device blind, it doesn't
sound different from the other, or at least not different enough to matter.
From my experience with these two amplifiers, that is clearly not the case.
The subtle differences DO matter. So then why wouldn't I be able to tell the
amps apart blind?
I suspect this has to do with how the brain processes information (so please
spare me the "you're imagining it" posts). As long as I am relaxed listening
to music, I can easily identify the subtle differences that separate these two
products. Once I begin to "problem-solve" to determine which amp I am
listening to, my brain switches gears. No longer am I relaxed, but I begin the
"decision-making" process to determine which amp I it is. I begin to search
for clues, isolating one part of the sound and then another, to see which
"amplifier-memory" it corresponds to. As my brain changes gears here, I do not
hear the subtle sounds at all, because I am too busy trying to focus on
"differences" that I can identify, not the music as a whole (this may explain
why pink noise works better in these tests in some cases). I believe this is
why it is much easier to distinguish two products blind with gross differences
such as speakers than it is with amps, CD players, cables, etc. which tend to
have more subtle differences (if any).
My overall conclusion from this experience is that if you hear differences
between components in casual listening, those differences are REAL. Furthermore
you should make your decisions based on those differences, not whether or not
the differences are evident in blind listening. Trust your ears.
Regards,
Mike
Arny Krueger
news:ajh7hk$3u9$1...@bourbaki.localdomain
> I recently purchased a new amplifier and have spent a lot of time
> listening to it and comparing it to my old amp. In this case,
> the two amplifiers sound much more similar than I would have
> expected, but I hear small differences between them that I
> consider important, especially for long term listening pleasure.
> Now when I sit and listen to music I relax and enjoy the
> experience. When I decide it's time to switch them, I listen to 3
> or 4 cuts that demonstrate specific musical qualities and take
> notes. After careful level matching (with a meter), I switch the
> amps and listen to the same music in the reverse order and take
> notes again. As I said, I have consistently noted important
> subtle differences in comparing the sound of the two amps playing
> music. One of the two amps would not satisfy me as much long
> term, particularly after having listened at length to the other
> one.
The problem with this approach is that its impossible to know for
sure what the source of the "...consistently noted important subtle
differences..." is. Are these subtle differences the result of
expectations, random perceptions, or the quality of the sound?
> However, I am absolutely convinced that if I were to listen to
> the amps blind, I would not be able to identify which one I was
> listening to.
Let's say that we would like to think that the author is a person who
is free of biases. Would a person who has no biases say something
like this? Would not a person who is free of biases base his beliefs
on actual real-world experiences, rather than seemingly baseless
speculations?
If the author can be this strongly convinced of something that he has
apparently no direct experience with like blind testing, isn't that
prima facae evidence that he is a strongly-biased person who comes to
far-reaching, closely held conclusions with no personal experience,
rational logic, or reliable evidence?
If we conclude that the author is strongly-biased person who comes to
far-reaching, closely held conclusions with no personal experience,
rational logic, or reliable evidence; what conclusions can we come to
about the first paragraph?
> The standard "objectivist" line is that if you
> can't identify the device blind, it doesn't sound different from
> the other, or at least not different enough to matter.
This is of course based on the idea that the best way to judge sound
quality is to just listen.
> From my
> experience with these two amplifiers, that is clearly not the
> case. The subtle differences DO matter. So then why wouldn't I
> be able to tell the amps apart blind?
Just a second. What kind of experience is the author talking about
here? Is the experience something that is reliable and iron-clad or
is it 100$ based on his pre-existing biases?
> I suspect this has to do with how the brain processes information
> (so please spare me the "you're imagining it" posts).
It looks very much to me that it has a great deal to do with the
expected behavior of strongly-biased person who comes to
far-reaching, closely held conclusions with no personal experience,
rational logic, or reliable evidence.
Gary Eickmeier
> I suspect this has to do with how the brain processes information (so please
> spare me the "you're imagining it" posts). As long as I am relaxed listening
> to music, I can easily identify the subtle differences that separate these two
> products. Once I begin to "problem-solve" to determine which amp I am
> listening to, my brain switches gears.
The oft repeated fallacy here is that in blind testing you must not
know which amp is which, or you cannot see the two devices. Au
contraire, you can set them up right next to each other, with a bright
spotlight on the appropriate one that activates when you switch to A
or B, listen to your heart's content for all the subtle differences
between them, all the while knowing precisely which you are listening
to and when.
Then, when you have "locked on" to some particular difference that you
know you can use to identify them blind, you switch between A and X,
then B and X, and determine which one X sounds like. If the difference
you heard sighted was real, it should be easy to also hear it blind.
Notice that you can still see both amps on the table or floor; you
just don't get the spotlight telling you that you are listening to A
when X is playing. The ONLY difference is that you are deprived of the
prior knowledge of which is playing when you select X, so you must
decide based on sound alone. Which is, of course, exactly the test we
are after.
Beautiful, isn't it?
Gary Eickmeier
Bob Marcus
> I recently purchased a new amplifier and have spent a lot of time listening to
> it and comparing it to my old amp. In this case, the two amplifiers sound much
> more similar than I would have expected, but I hear small differences between
> them that I consider important, especially for long term listening pleasure.
> Now when I sit and listen to music I relax and enjoy the experience. When I
> decide it's time to switch them, I listen to 3 or 4 cuts that demonstrate
> specific musical qualities and take notes. After careful level matching (with
> a meter), I switch the amps and listen to the same music in the reverse order
> and take notes again. As I said, I have consistantly noted important subtle
> differences in comparing the sound of the two amps playing music. One of the
> two amps would not satisfy me as much long term, particularly after having
> listened at length to the other one.
We all have our favorite ways of choosing audio gear. Yours is as
valid as any other--as long as you don't go making technical or
scientific claims based on it.
>
> However, I am absolutely convinced that if I were to listen to the amps blind,
> I would not be able to identilfy which one I was listening to.
You haven't said which amps, but in general I'd agree with you here.
On the other hand, you won't know until you try.
>The standard
> "objectivist" line is that if you can't identify the device blind, it doesn't
> sound different from the other, or at least not different enough to matter.
> From my experience with these two amplifiers, that is clearly not the case.
C'mon, Mike. You've been around here long enough to know better than
that. It may not *seem* the case to you, but that's as far as you can
go based on the experience you've recounted here.
> The subtle differences DO matter. So then why wouldn't I be able to tell the
> amps apart blind?
>
> I suspect this has to do with how the brain processes information (so please
> spare me the "you're imagining it" posts).
Correction: It's not that you *are* imagining it. It's that you
*might* be imagining it. Only a comparison that eliminates potential
biases can determine whether that is so.
>As long as I am relaxed listening
> to music, I can easily identify the subtle differences that separate these two
> products. Once I begin to "problem-solve" to determine which amp I am
> listening to, my brain switches gears. No longer am I relaxed, but I begin the
> "decision-making" process to determine which amp I it is.
Then don't do it as an ABX test. Try this: Follow your same listening
approach, but do it blind. At the end of each session, decide which
amp you like best. You don't have to think about which is which at
all; just determine a preference.
If you keep coming up with the same preference each time, then I'd say
those amps probably do sound different. But if your preferences aren't
so consistent, then I'd say your original preference was based on
something other than the sound.
bob
Stewart Pinkerton
>I recently purchased a new amplifier and have spent a lot of time listening to
>it and comparing it to my old amp. In this case, the two amplifiers sound much
>more similar than I would have expected, but I hear small differences between
>them that I consider important, especially for long term listening pleasure.
Ok, that's a claim based on sighted 'listening'.
>Now when I sit and listen to music I relax and enjoy the experience. When I
>decide it's time to switch them, I listen to 3 or 4 cuts that demonstrate
>specific musical qualities and take notes. After careful level matching (with
>a meter), I switch the amps and listen to the same music in the reverse order
>and take notes again. As I said, I have consistantly noted important subtle
>differences in comparing the sound of the two amps playing music. One of the
>two amps would not satisfy me as much long term, particularly after having
>listened at length to the other one.
OK, that's another claim based on sighted 'listening'. Note please
that you *are* performing a conscious comparison here, so let's not
have any bull about how this is not how you normally listen to music.
You have clearly stated that you are playing the same pieces of music
on each amp, matching levels, and listening for differences.
>However, I am absolutely convinced that if I were to listen to the amps blind,
>I would not be able to identilfy which one I was listening to. The standard
>"objectivist" line is that if you can't identify the device blind, it doesn't
>sound different from the other, or at least not different enough to matter.
Correct.
From my experience with these two amplifiers, that is clearly not the case.
>The subtle differences DO matter. So then why wouldn't I be able to tell the
>amps apart blind?
Because it is *not* 'clearly not the case', since you are not
comparing sound quality by just listening, you are including a whole
baggage of *knowledge* of which amp is playing. You have *no* definite
knowledge that these 'subtle differences' really exist in the physical
soundfield.
>I suspect this has to do with how the brain processes information (so please
>spare me the "you're imagining it" posts). As long as I am relaxed listening
>to music, I can easily identify the subtle differences that separate these two
>products.
Yes, but do those 'subtle differences' have anyhting to do with the
*sound* quality of the amplifiers? Besides, you've already admitted
that you are doing comparison tests, rather than just listening to
music.
> Once I begin to "problem-solve" to determine which amp I am
>listening to, my brain switches gears. No longer am I relaxed, but I begin the
>"decision-making" process to determine which amp I it is. I begin to search
>for clues, isolating one part of the sound and then another, to see which
>"amplifier-memory" it corresponds to.
And this would be different in a blind test, how? You have already
admitted to careful level matching and listening to the same pieces of
music on each amp, so what's the problem with doing *exacyly* the same
thing, with the sole difference that you don't *know* which amp is
playing.
> As my brain changes gears here, I do not
>hear the subtle sounds at all, because I am too busy trying to focus on
>"differences" that I can identify, not the music as a whole (this may explain
>why pink noise works better in these tests in some cases).
Doesn't seem to cause you a problem in your *sighted* comparison, now
does it?
> I believe this is
>why it is much easier to distinguish two products blind with gross differences
>such as speakers than it is with amps, CD players, cables, etc. which tend to
>have more subtle differences (if any).
No one uses ABX with speakers, simply because the differences *are*
gross!
>My overall conclusion from this experience is that if you hear differences
>between components in casual listening, those differences are REAL.
Your conclusion is flawed, and exhibits pretty much the same
prejudices that allow you to 'hear' those differences in *sighted*
comparisons.
> Furthermore
> you should make your decisions based on those differences, not whether or not
>the differences are evident in blind listening. Trust your ears.
You say 'trust your ears', and yet you refuse to do it yourself!
--
Stewart Pinkerton | Music is Art - Audio is Engineering
you_know_who~
not even be in the amps..
I have found that a combination of components (mixture of amps and
pre-amps, for example) will cause subtle differences which are quite
hard to distinguish. Unless the amps are seen as identical by the
preamp (which is unlikely) some slight sound difference can occur.
I would agree that there are small tonal differences which exist but
cannot be heard in direct A/B comparisons. I am not so sure though
that they really matter all that much. Since you state that you
could listen to both amps and enjoy them, the marginal difference
may, indeed, not matter all that much.
I would never say that you are imagining things (how would I know?).
I am quite sure though that almost all of us would be eager to
justify an expensive purchase, especially when one acknowledges that
the purchase did not lead to any demonstrable improvement.
--
You Know Who~
"No matter how cynical you get,
it is impossible to keep up." -- Lily Tomlin
-----------------------------------------------------
Cat stuff, Win Help, Audio & Purgatory
@ http://You_Know_Who.home.att.net/
`````````````````````````````````````````````
"Mkuller" <mku...@aol.com> wrote in message
news:ajh7hk$3u9$1...@bourbaki.localdomain...
Cannot Understand Why You Dis AOL
From: mku...@aol.com (Mkuller)
>Date: 08/15/2002 5:49 PM Eastern Daylight Time
>Message-id: <ajh7hk$3u9$1...@bourbaki.localdomain>
>
>I recently purchased a new amplifier and have spent a lot of time listening
>to
>it and comparing it to my old amp. In this case, the two amplifiers sound
>much
>more similar than I would have expected, but I hear small differences between
>them that I consider important, especially for long term listening pleasure.
>Now when I sit and listen to music I relax and enjoy the experience. When I
>decide it's time to switch them, I listen to 3 or 4 cuts that demonstrate
>specific musical qualities and take notes. After careful level matching
>(with
>a meter), I switch the amps and listen to the same music in the reverse order
>and take notes again. As I said, I have consistantly noted important subtle
>differences in comparing the sound of the two amps playing music. One of the
>two amps would not satisfy me as much long term, particularly after having
>listened at length to the other one.
>
>However, I am absolutely convinced that if I were to listen to the amps
>blind,
>I would not be able to identilfy which one I was listening to. The standard
>"objectivist" line is that if you can't identify the device blind, it doesn't
>sound different from the other, or at least not different enough to matter.
From my experience with these two amplifiers, that is clearly not the case.
<sigh> No, it *is not at all* clearly the case.
By your own admission, your experience of the two amps has been
exclusively sighted. Therefore you haven't even tested the
'objectivist' hypothesis, much less refuted it.
That you could make such an absolutely fundamental logical error,
right up front, does not bode well for the rest of this thread. And
it renders your theorizing in the rest of your post moot.
_______
-S.
Klaus Rampelmann
>My overall conclusion from this experience is that if you
>hear differences between components in casual listening,
>those differences are REAL.
Allow me to cite from J. Risch, "A user friendly methodology for
subjective listening tests", 91th AES convention, preprint 3178 :
"It must be kept in mind, that the test results, any subjective test
results, are only valid for those participants, on that particular
source system in that particular room with the particular music
selections used."
Which clearly means that your findings, or the findings of any other
audiophile, are of no relevance for others. If you hear a difference
between amps A and B, fine. If I don't hear a difference between these
same amps, fine. You will buy the "better sounding" amp, I will stay
with what I have. We both have proven our case and can sleep well.
That's great for both of us.
Have fun on audioplanet.
Klaus
Nousaine
....snip of all points in which I'm in complete agreement ........
>
>No one uses ABX with speakers, simply because the differences *are*
>gross!
>
Actually I sometimes use an ABX comparator and a visually opague but
acoustically transparent cloth screen to help reduce bias when
making speaker comparisons. You'd be surprised at how much such a
presentation method reduces scoring variance.
BEAR
> <snip>
>
> I suspect this has to do with how the brain processes information (so please
> spare me the "you're imagining it" posts). As long as I am relaxed listening
> to music, I can easily identify the subtle differences that separate these two
> products. Once I begin to "problem-solve" to determine which amp I am
> listening to, my brain switches gears.
> Regards,
> Mike
<snip>
It seems to me that the main issue here is not "blind" testing, but
the rate of change between two sources.
I believe that the brain (well some brains) use different processes
to "decode" acoustic information depending upon the situation. I
hypothesize that in terms of "relaxed long term listening" that the
difference heard is often due to the *ease* that the brain has in
decoding the sound into recognizable parts. In other words, the
*less* work that has to be done to identify and recognize the
acoustic information, the "better" the system sounds...
Clearly, looked at from this point of view, you can logically infer
that there is more than one simple "level" of "hearing" to the sound
presented. It would be entirely possible for your brain to decode (an
essentially real time process) the sound enough to be able to
recognize what it is on some relatively basic level, but not beyond.
IF this situation was caused by any one of a number of external or
internal factors then truly "all amplifiers sound the same."
So, the situation which Mkuller speaks of should not be poo-poo'd
quite so quickly in my estimation. As far as I can see what he is
speaking of relates directly to the situation I have briefly outlined
above, wherein his "relaxed" method may in fact permit *him* to more
effectively process the acoustic information more fully and
completely - thus revealing some subtle differences.
* * *
By way of an anecdotal example:
I've got a Dutoit/Bolet recording of some Tchaikovsky/Rachmaninov
(London 421 181-2) wherein someone whacks a music stand a few minutes
into the recording. About 90% of the time, the music goes by without
noticing the whack at all. In fact in a few cases where I wanted to
*demonstrate* this sound to someone, I had trouble *finding* it at
all.
In some part this is why "magic" works. The mind can be "led" and
distracted and mis-directed without too much trouble.
Hope this makes sense, even to the "objectivists" out there...
_-_-bear
--
_-_-bearlabs
http://www.bearlabsUSA.com
- Silver Lightning Interconnects -
John S.
news:aji93l$o3g$1...@bourbaki.localdomain...
> On Thu, 15 Aug 2002 21:49:16 GMT, mku...@aol.com (Mkuller) wrote:
>
> >I recently purchased a new amplifier and have spent a lot of time
listening to
> >it and comparing it to my old amp. In this case, the two amplifiers
sound much
> >more similar than I would have expected, but I hear small differences
between
> >them that I consider important, especially for long term listening
pleasure.
>
> Ok, that's a claim based on sighted 'listening'.
Before buying a new amp I took my older Adcom 555II in to my local shop and
compared it to a new Rotel. I knew which was hooked up but that was the
point, to know the differences. The Rotel was as grainy on the highs (on
some passages) as my Adcom but, on the lows it was a little further extended
and detailed. I could hear the foot pedals on a pipe organ piece with the
Rotel. I could barely discern it with the Adcom.
Not being compeltely happy with the idea of spending $1k for a amp with only
sllight improvements, I passed. Their next step up was Mark Levinson gear
which started at $4k and went up. Too much for my blood.
I decided to buy a McCormack amp (from right here in Virginia) after reading
many reviews and corresponding with sevearl people that have owned or still
own them. I could not be happier with my decision. The DNA-125 on the above
mentioned passage displayes the full foot pedal base of the pipe organ. The
highs are smooth. All the bad things that I despised with the Adcom unit
once I switch to the Revel M20's are gone. My other speakers did not reveal
the source as the Revels do. I was forced to make an upgrade where none was
planned.
At the shop where the Rotel/Adcom comaprison took place, I used their cables
switching from theirs to mine. Exact same cables were used, not another
similar pair but the exact ones. Power cord was into same outlet on their
protection unit.
Compared to here at home, I have the same cd player we used but different
cables. They have high end stuff. I have el cheapo stuff. My speaker wires
are home made from double runs of 14 guage wiring ( 4 to each speaker,
briaded not solid core). My interconnects are also cheapies from
www.knukonceptz.com . I use their shielded twisted pair model. (Flame away
at using cheap stuff if you like). Oh, I also use stock power cords sicne I
have yet to see internal power wiring greater than 18 perhaps 16 guage and
none of it was shielded. I believe most power cords are 18g. 12g would be
nice for really long runs but most power cords are only a meter or 2 long
anyway.
Mkuller
>OK, that's another claim based on sighted 'listening'. Note please
>that you *are* performing a conscious comparison here, so let's not
>have any bull about how this is not how you normally listen to music.
>You have clearly stated that you are playing the same pieces of music
>on each amp, matching levels, and listening for differences.
>
critically and taking notes, those are similar activities which I suspect
utilize the same brain functions. It's when one switches activities from
listening to making a decision which requires "matching/identifiing/choosing"
that things change. Somehow this function seems to interfere with recalling
memories of subtler audible cues. More often than not when the audible cues
are subtle,
blind listening results in "no difference heard". This is opposed to sighted
listening (no decision-making is required) where subtle audible differences
are easily heard.
>Yes, but do those 'subtle differences' have anyhting to do with the
>*sound* quality of the amplifiers? Besides, you've already admitted
>that you are doing comparison tests, rather than just listening to
>music.
>
>> Once I begin to "problem-solve" to determine which amp I am
>>listening to, my brain switches gears. No longer am I relaxed, but I begin
>the
>>"decision-making" process to determine which amp I it is. I begin to search
>>for clues, isolating one part of the sound and then another, to see which
>>"amplifier-memory" it corresponds to.
>
>And this would be different in a blind test, how? You have already
>admitted to careful level matching and listening to the same pieces of
>music on each amp, so what's the problem with doing *exacyly* the same
>thing, with the sole difference that you don't *know* which amp is
>playing.
>
Because this requires an activity which is different from "listening" and
casually comparing. It requires making a decision -
matching/identifiing/choosing.
Regards,
Mike
Howard Ferstler
>
> I recently purchased a new amplifier and have spent a lot of time listening to
> it and comparing it to my old amp. In this case, the two amplifiers sound much
> more similar than I would have expected, but I hear small differences between
> them that I consider important, especially for long term listening pleasure.
> Now when I sit and listen to music I relax and enjoy the experience. When I
> decide it's time to switch them, I listen to 3 or 4 cuts that demonstrate
> specific musical qualities and take notes. After careful level matching (with
> a meter), I switch the amps and listen to the same music in the reverse order
> and take notes again. As I said, I have consistantly noted important subtle
> differences in comparing the sound of the two amps playing music. One of the
> two amps would not satisfy me as much long term, particularly after having
> listened at length to the other one.
>
> However, I am absolutely convinced that if I were to listen to the amps blind,
> I would not be able to identilfy which one I was listening to.
This tells us quite a bit. Obviously, your sighted
comparisons involved your feelings about each amp - feelings
that were unrelated to how they actually sounded. And there
is no way for you to tell if those feelings were influencing
what you claim you heard.
> The standard
> "objectivist" line is that if you can't identify the device blind, it doesn't
> sound different from the other, or at least not different enough to matter.
That is my way of looking at it precisely. What else would
anyone want?
> From my experience with these two amplifiers, that is clearly not the case.
> The subtle differences DO matter.
But you just confessed that you might not be able to tell
them apart during a blind comparison. Any subtle differences
you think were there during your sighted evaluations have
the potential to basically be no more than fabrications
based upon how you thought each amp should sound. You have
no way to prove to anyone, including yourself, if the
differences you think you heard were really audible.
> So then why wouldn't I be able to tell the
> amps apart blind?
Because they they sounded the same.
> I suspect this has to do with how the brain processes information (so please
> spare me the "you're imagining it" posts). As long as I am relaxed listening
> to music, I can easily identify the subtle differences that separate these two
> products. Once I begin to "problem-solve" to determine which amp I am
> listening to, my brain switches gears. No longer am I relaxed, but I begin the
> "decision-making" process to determine which amp I it is. I begin to search
> for clues, isolating one part of the sound and then another, to see which
> "amplifier-memory" it corresponds to.
You could still do this blind. Why does this laid-back test
of yours have to be done sighted? Do the same thing without
knowing which amp is playing. Do them in the same relaxed
manner, and see if the differences show up.
> As my brain changes gears here, I do not
> hear the subtle sounds at all, because I am too busy trying to focus on
> "differences" that I can identify, not the music as a whole (this may explain
> why pink noise works better in these tests in some cases). I believe this is
> why it is much easier to distinguish two products blind with gross differences
> such as speakers than it is with amps, CD players, cables, etc. which tend to
> have more subtle differences (if any).
No matter what you think about the effectiveness of sighted
comparisons, there is no way to tell if you are really
hearing differences unless you do the comparison blind. You
have no way of solidly knowing if you are fooling yourself
during sighted comparisons.
> My overall conclusion from this experience is that if you hear differences
> between components in casual listening, those differences are REAL.
They may be to you, and if this makes you feel better, fine.
However, one would think that a serious audio enthusiast
would want to know more about amp sound than what can be
gained from doing a series of sighted comparisons that
basically have the potential to satisfy nothing more than
one's preconceptions.
> Furthermore
> you should make your decisions based on those differences, not whether or not
> the differences are evident in blind listening. Trust your ears.
Which is why the comparisons should be done blind if one
REALLY wants to know if differences are audible.
Your procedure may make you feel better and it may make you
a happier audio enthusiast to do things that way. However,
you will never know for sure if what you think you are
hearing is really audible if you do not do the comparing
blind. Not doing them blind always leaves the question
hanging.
Howard Ferstler
Mkuller
>> However, I am absolutely convinced that if I were to listen to the amps
>blind,
>> I would not be able to identilfy which one I was listening to.
>
>This tells us quite a bit. Obviously, your sighted
>comparisons involved your feelings about each amp - feelings
>that were unrelated to how they actually sounded.
Sorry Howard, you have no way to know if my feelings were involved or if I
based my conclusions soley on audible differences. Are you a mind-reader?
As an experienced listener, with no preconceptions, I believe I can put any
biases aside. In fact my listening results were not at all what I would have
predicted. I ended up preferring my old tube amps to a new, highly regarded
(and nearly identical sounding) solid state model.
>> The standard
>> "objectivist" line is that if you can't identify the device blind, it
>doesn't
>> sound different from the other, or at least not different enough to matter.
>
>That is my way of looking at it precisely. What else would
>anyone want?
Most audiophiles want to base their decisions on ALL the differences they hear.
If some of those subtle differences disappear or are not able to be recalled
during the blind listening decision-making-process, then this procedure is not
valid for making decisions about audio equipment using music.
>
>> From my experience with these two amplifiers, that is clearly not the case.
>> The subtle differences DO matter.
>
>But you just confessed that you might not be able to tell
>them apart during a blind comparison. Any subtle differences
>you think were there during your sighted evaluations have
>the potential to basically be no more than fabrications
>based upon how you thought each amp should sound.
I'll give you that is a "potential". An experienced listener (certainly an
audio reviewer not investing his own money) should be able to separate the two.
>You have
>no way to prove to anyone, including yourself, if the
>differences you think you heard were really audible.
>
So it's proof you're after. How can you prove those subtle differences you
hear in sighted listening that cannot be identified or recalled from your
memory during the blind "decision-making process" aren't really there, and not
obscured by how your brain processes information?
>> So then why wouldn't I be able to tell the
>> amps apart blind?
>
>Because they they sounded the same.
>
No they had subtle, but important audible differences.
>> I suspect this has to do with how the brain processes information (so
>please
>> spare me the "you're imagining it" posts). As long as I am relaxed
>listening
>> to music, I can easily identify the subtle differences that separate these
>two
>> products. Once I begin to "problem-solve" to determine which amp I am
>> listening to, my brain switches gears. No longer am I relaxed, but I begin
>the
>> "decision-making" process to determine which amp I it is. I begin to
>search
>> for clues, isolating one part of the sound and then another, to see which
>> "amplifier-memory" it corresponds to.
>
>You could still do this blind. Why does this laid-back test
>of yours have to be done sighted? Do the same thing without
>knowing which amp is playing. Do them in the same relaxed
>manner, and see if the differences show up.
>
From my experience, I don't believe you can "listen in a relaxed manner" so
that you are aware of subtle details and make a decision about whether X is A
or B at the same time. The matching/identifying/choosing process becomes the
priority - not the relaxed listening. They are incompatible when the audible
cues are subtle.
>> As my brain changes gears here, I do not
>> hear the subtle sounds at all, because I am too busy trying to focus on
>> "differences" that I can identify, not the music as a whole (this may
>explain
>> why pink noise works better in these tests in some cases). I believe this
>is
>> why it is much easier to distinguish two products blind with gross
>differences
>> such as speakers than it is with amps, CD players, cables, etc. which tend
>to
>> have more subtle differences (if any).
>
>No matter what you think about the effectiveness of sighted
>comparisons, there is no way to tell if you are really
>hearing differences unless you do the comparison blind. You
>have no way of solidly knowing if you are fooling yourself
>during sighted comparisons.
>
And based on your comments, you have no way of knowing if you are losing subtle
audible information during your blind tests and coming to incorrect conclusions
based on them.
>> My overall conclusion from this experience is that if you hear differences
>> between components in casual listening, those differences are REAL.
>
>They may be to you, and if this makes you feel better, fine.
>However, one would think that a serious audio enthusiast
>would want to know more about amp sound than what can be
>gained from doing a series of sighted comparisons that
>basically have the potential to satisfy nothing more than
>one's preconceptions.
>
>> Furthermore
>> you should make your decisions based on those differences, not whether or
>not
>> the differences are evident in blind listening. Trust your ears.
>
>Which is why the comparisons should be done blind if one
>REALLY wants to know if differences are audible.
>
Differences that are audible sighted are not always audible blind. Agreed?
Now your explanation is preconception and bias and I reject that. I say it's
due to the way the brain functions.
>Your procedure may make you feel better and it may make you
>a happier audio enthusiast to do things that way. However,
>you will never know for sure if what you think you are
>hearing is really audible if you do not do the comparing
>blind. Not doing them blind always leaves the question
>hanging.
>
Doing them blind always leaves the question "What am I missing here? Why can't
I identify it now when the differences were so obvious sighted?"
Regards,
Mike
Gary Eickmeier
> Doing them blind always leaves the question "What am I missing here? Why can't
> I identify it now when the differences were so obvious sighted?"
Time for another analogy.
Suppose you meet a guy who claims to be the most brilliant wine taster
in the world. You question his boast, and ask for a demonstration. You
bet him $100 he can't tell a California Pinot Noir from a French one.
So he sits down with a bottle of each, with the labels still on them,
pours a glass from each, and tastes away. He pronounces that the one
that came from the French bottle is the French wine, and the one that
came from the California label is from California.
Do you pay him?
Why not? He agrees with you that he can tell many more subtle
differences tasting them sighted vs blind. So what's your problem?
Gary Eickmeier
Mkuller
>The oft repeated fallacy here is that in blind testing you must not
>know which amp is which, or you cannot see the two devices. Au
>contraire, you can set them up right next to each other, with a bright
>spotlight on the appropriate one that activates when you switch to A
>or B, listen to your heart's content for all the subtle differences
>between them, all the while knowing precisely which you are listening
>to and when.
>
>Then, when you have "locked on" to some particular difference that you
>know you can use to identify them blind, you switch between A and X,
>then B and X, and determine which one X sounds like. If the difference
>you heard sighted was real, it should be easy to also hear it blind.
>Notice that you can still see both amps on the table or floor; you
>just don't get the spotlight telling you that you are listening to A
>when X is playing. The ONLY difference is that you are deprived of the
>prior knowledge of which is playing when you select X, so you must
>decide based on sound alone. Which is, of course, exactly the test we
>are after.
>
>Beautiful, isn't it?
My premise is that once you have to "match/identify/select" which one
X is, you are no longer engaged in relaxed listening, but you are
using a different brain process that gets in the way of your memory
of subtle audio cues. That's why many of the subtle audible
differences which are evident in sighted listening are not
identifiable when listening and choosing blind.
Regards,
Mike
Mkuller
>> I believe this is
>>why it is much easier to distinguish two products blind with gross
>differences
>>such as speakers than it is with amps, CD players, cables, etc. which tend
>to
>>have more subtle differences (if any).
>
>No one uses ABX with speakers, simply because the differences *are*
>gross!
Following your (biased) "logic", how do you know that you are really
"hearing" those differences, gross or not, if you're listening is
sighted? Why wouldn't you apply the same standards to all of your
listening?
Regards,
Mike
jjn...@sonic.net
> Because this requires an activity which is different from "listening" and
> casually comparing. It requires making a decision -
> matching/identifiing/choosing.
If you were a music student, would you feel compelled to 'cheat' on
ear training exams?
Do you think that musicians who have the skills acquired through ear
training are more 'deaf'and/or produce worse music than those who do
not?
Jonathan Babiak
> <cut>
> So, the situation which Mkuller speaks of should not be poo-poo'd
> quite so quickly in my estimation. As far as I can see what he is
> speaking of relates directly to the situation I have briefly outlined
> above, wherein his "relaxed" method may in fact permit *him* to more
> effectively process the acoustic information more fully and
> completely - thus revealing some subtle differences.
>
> In some part this is why "magic" works. The mind can be "led" and
> distracted and mis-directed without too much trouble.
>
> Hope this makes sense, even to the "objectivists" out there...
>
> _-_-bear
I'd call myself a "hardcore" objectivist, which admittedly is probably
not the best viewpoint, but I think I understand you perfectly. The
listener is a very important link in the chain. The listener's state
of mind, if you will, can influence the listener's perception.
Depending on this state of mind at any given time, the listener may
perceive differences that exist in the equipment to which the listener
is listening, or the differences may be only in the mind of the
listener. The listener's state of mind at the time of the "test" must
be considered before concluding if the perceived differences are such
that another listener will perceive the same differences when
listening to the same equipment. Even if the differences exist in the
equipment, another listener who is not in the same state of mind as
the first listener won't perceive the differences, even if the
differences are "objectively" measurable.
I think that one problem here may be in the use of the term "real"
differences. An objectivist will claim that the only "real"
differences are those differences that exist entirely "within" the
equipment, including perhaps the listening room. Objectivists reject
the notion that any *perceived* difference must also be "real," until
it can be shown that the perceived difference can be detected by
testing that satisfies the typical objectivist.
And if my understanding of your proposition is not accurate, just say
so. I'm not trying to be confrontational. Do you think a difference
that a listener honestly perceives is necessarily a "real" difference,
or would you define "real" difference some other way?
ps All typographical errors become the property of the finder. No
refunds.
ludovic mirabel
Dear Mr. Rampelmann, you put me in your debt again. I have used your
excellent bibliography of component comparison tests on several
occasions. Now you found this fascinating quote.
It is great to see someone say in JAES what I've been saying here for
two years (and got brickbats for)
There is no "objective" "test" sighted or blind for comparing how
audio components differ when reproducing music. There is only a
multitude of testers with their particular innate abilities, their
ears, their infinitely variable musical experience and interests in
their particular rooms with their particular component collection.
Try and collect a large ebough panel of 40 year old half male , half
female chamber music lovers, with equal ABX training and similar
*tested* excellent ABX ability (a major problem in itself), similar
hearing ability in a representative?! audio room with
representative?! audio equipment. Such a panel will help others in an
identical ?! situation but hardly anyone else. All the rest of us will
have to fend for ourselves - helped by the gurus one trusts, ditto
friends and wives and in the end our own listening.
Ludovic Mirabel
Gary Eickmeier
> My premise is that once you have to "match/identify/select" which one
> X is, you are no longer engaged in relaxed listening, but you are
> using a different brain process that gets in the way of your memory
> of subtle audio cues. That's why many of the subtle audible
> differences which are evident in sighted listening are not
> identifiable when listening and choosing blind.
I'm sorry Mike, but I can't buy that. I told you that you are allowed
to listen sighted until you hear some particular, identifiable
difference. When you do finally hear such a difference, all that is
left is to identify X in the blind portion to prove that you really
did find something.
What you are talking about is the testing part alone, as if you could
not listen sighted and relaxed for as long as you wanted before going
to X. After all, you say that you can hear a "difference" under
sighted conditions, which means you must be switching between devices
and listening for differences, right? Why would you say that doing
that is any less stressful on your poor little brain than switching
between A and X? Absolutely no difference except that you don't know
in advance which component X is.
Gary Eickmeier
BEAR
> BEAR <bear...@netzero.net> wrote:
> > <cut>
> > So, the situation which Mkuller speaks of should not be poo-poo'd
> > quite so quickly in my estimation. As far as I can see what he is
> > speaking of relates directly to the situation I have briefly outlined
> > above, wherein his "relaxed" method may in fact permit *him* to more
> > effectively process the acoustic information more fully and
> > completely - thus revealing some subtle differences.
> >
> > <cut even more, but please don't take it personally!>
> > In some part this is why "magic" works. The mind can be "led" and
> > distracted and mis-directed without too much trouble.
> >
> > Hope this makes sense, even to the "objectivists" out there...
> >
> > _-_-bear
>
>
<snip>
> The listener's state of mind at the time of the "test" must
> be considered before concluding if the perceived differences are such
> that another listener will perceive the same differences when
> listening to the same equipment.
Which, of course is very difficult if not impossible to quantify or
measure.
> Even if the differences exist in the
> equipment, another listener who is not in the same state of mind as
> the first listener won't perceive the differences, even if the
> differences are "objectively" measurable.
This may be true as well. This statement is more or less the inverse
of what I hypothesized.
>
>
> I think that one problem here may be in the use of the term "real"
> differences.
I'm pretty sure I did not use that term. There are very real and
measurable differences between two channels of a very high quality
stereo amplifier, so "real" and audible are not the same thing.
Therein lies the entire controversial topic in a nutshell.
> An objectivist will claim that the only "real"
> differences are those differences that exist entirely "within" the
> equipment, including perhaps the listening room. Objectivists reject
> the notion that any *perceived* difference must also be "real," until
> it can be shown that the perceived difference can be detected by
> testing that satisfies the typical objectivist.
I think you've mis-stated the objectivist's position - but I'll leave
that to "them" to correct the concepts...
>
>
> And if my understanding of your proposition is not accurate, just say
> so. I'm not trying to be confrontational. Do you think a difference
> that a listener honestly perceives is necessarily a "real" difference,
> or would you define "real" difference some other way?
I would not attempt to define "real" differences for the purpose of
this discussion, and it plays no significant role whatsoever in the
discussion I presented. In the situation I presented, there may be no
difference in the sound that is presented per se - rather what I
tried to talk about are the factors that effect the perception of
said sound that are the result of the way that people hear and
listen, and process the sound in the brain, thereby making "sense" of
it, which is perception.
To continue along the logic which my hypothesis presents, each and
every time someone listens (to the same sound, via their hi-fi in
this case) they hear an entirely different thing. Similar to be sure,
maybe even virtually identical in some instances, but usually varied
and different each and every time. Imho, ultimately the perceived
quality of a given system/environment is actually an abstract
quantification of the *effort* required by the brain to figure out
and make sense of the sound - less effort = better system, more
effort=less good system.
Along those same lines, we could suggest a situation where a
"trained" listener (effective and complex brain algorithms for
decoding sound) *could* percieve *more* information from a "bad
system" than a naive or otherwise "untrained" listener might from a
"good system."
My personal experience and perhaps some reading on the subject
suggests that to a great extent individuals are "hardwired" to
perceive certain things & types of sound(s) more readily than other
sounds. To use an analogy, a compiled program tends to run faster on
a given processor than one that is interpreted - since sound is a
real time input, the processing that the brain can do on the incoming
"data" stream is finite. So, it seems logical and likely that this
plays a great role in what you hear and how you hear "into" the
sound...
For an example to think about - consider the case where you flip on
your radio in the car, and a tune pops out in the middle, in
progress. In some cases, you just don't recognize it immediately.
A few seconds into it, you figure out the context and recognize the
tune, and know what it is and where you are. But before that, it is
foreign and strange. I know that's happened to me more than once. I
suspect that it happens to everyone.
Perception is not a constant, to say the least. Humans are a
variable.
regards,
Mkuller
>>
>> My premise is that once you have to "match/identify/select" which one
>> X is, you are no longer engaged in relaxed listening, but you are
>> using a different brain process that gets in the way of your memory
>> of subtle audio cues. That's why many of the subtle audible
>> differences which are evident in sighted listening are not
>> identifiable when listening and choosing blind.
>
>I'm sorry Mike, but I can't buy that. I told you that you are allowed
>to listen sighted until you hear some particular, identifiable
>difference. When you do finally hear such a difference, all that is
>left is to identify X in the blind portion to prove that you really
>did find something.
>
>What you are talking about is the testing part alone, as if you could
>not listen sighted and relaxed for as long as you wanted before going
>to X. After all, you say that you can hear a "difference" under
>sighted conditions, which means you must be switching between devices
>and listening for differences, right? Why would you say that doing
>that is any less stressful on your poor little brain than switching
>between A and X? Absolutely no difference except that you don't know
>in advance which component X is.
>
to view it with an open mind. Listening to A and B is one thing, trying to
decide which X is, is altogether a different activity which requires use of a
different brain process. Call it "problem-solving" or "making a choice",
whatever, but it is very different from listening and noting differences. To
decide which choice X is, you must match its audible cues to the ones in your
memory, identify it as A or B and make a conscious choice. This requires
higher level brain activity than merely listening. This higher level brain
activity interferes with recalling subtle audible cues.
So in your view, you listen to A and B and hear easily identifiable, but subtle
cues which distinguish them, then switch to X and can't identify it accurately.
What is the explanation?
Regards,
Mike
S888Wheel
>of yours have to be done sighted? Do the same thing without
>knowing which amp is playing. Do them in the same relaxed
>manner, and see if the differences show up.
>
*If* the issue is that test conditions change the state of mind and mask
otherwise audible differences, no amount of relaxation or effort to make the
blind test seem more laid back like the sighted test will help. You cannot fool
a test subject into thinking he or she isn't listening under test conditions
and being tested as DBTs have been conducted. Here is a similar example of
state of mind affecting ones performance in a way that cannot be as easily
solved as one would at first think.. Actors spend years training to learn to
say lines from a script just the way we speak in our day to day lives. It is
easy for us to speak our minds and sound natural, without training it is nearly
impossible to speak memorized lines and make them sound natural. Why? Because
we are using different parts of our brains. Many an amateur actor thought they
could will their way around this reality only to be embarrassed. *If* the mere
implimentation of DBTs has the same effect as stated above and changes the very
way the brain is functioning while in the act of percieving sound you have a
problem that is not easily solved. Perhaps what would be an interesting test
would be to do blind tests of the claims of "subjectivists" that are disputed
by "objetivists" where the listener doesn't know that he or she is being
tested. If this has been done I apologize for my ignorance but I have never
seen any reports on such a test.
I would however certainly like to see Mkuller take the test before making any
assumptions about what results he will get.
Stewart Pinkerton
Simple. If you have any doubts, you do a blind test. When you've
scored 100% so many times that it's become boring, you don't waste
your time any more.
Howard Ferstler
>
> >Gary Eickmeier wrote:>
> >The oft repeated fallacy here is that in blind testing you must not
> >know which amp is which, or you cannot see the two devices. Au
> >contraire, you can set them up right next to each other, with a bright
> >spotlight on the appropriate one that activates when you switch to A
> >or B, listen to your heart's content for all the subtle differences
> >between them, all the while knowing precisely which you are listening
> >to and when.
> >
> >Then, when you have "locked on" to some particular difference that you
> >know you can use to identify them blind, you switch between A and X,
> >then B and X, and determine which one X sounds like. If the difference
> >you heard sighted was real, it should be easy to also hear it blind.
> >Notice that you can still see both amps on the table or floor; you
> >just don't get the spotlight telling you that you are listening to A
> >when X is playing. The ONLY difference is that you are deprived of the
> >prior knowledge of which is playing when you select X, so you must
> >decide based on sound alone. Which is, of course, exactly the test we
> >are after.
> >
> >Beautiful, isn't it?
> My premise is that once you have to "match/identify/select" which one
> X is, you are no longer engaged in relaxed listening, but you are
> using a different brain process that gets in the way of your memory
> of subtle audio cues.
This is speculative. You are basically setting yourself up
as a reference standard and then declaring that any
procedure not able to validate that standard is somehow
flawed. Most researchers would have it exactly backwards
from what you say: any sighted procedure that is invalidated
by a blind comparison has been proven to be influenced by
factors unrelated to the actual sound of the components
being compared.
> That's why many of the subtle audible
> differences which are evident in sighted listening are not
> identifiable when listening and choosing blind.
Again, there is no way to validate your claim. On the other
hand, a blind comparison can have its results validated by
other comparisons done by other individuals.
In your case, all sorts of different results may happen. One
guy may prefer the sound of one amp and another guy may
prefer the sound of the other amp. This happens all the time
with sighted comparisons. However, nothing like that happens
when blind comparisons are repeatedly done and most (or most
likely all if we are talking about wires) of the
participants can hear no differences at all.
While you may be comfortable setting yourself up as a
reference standard for what and what does not sound good,
and while that may be fine for you when you choose
components for yourself, your subjective, sighted abilities
have no relevance whatsoever outside of your own situation
and belief system.
What's more, there is no way you can absolutely prove even
to yourself that your technique is valid. You may hope that
it is valid, but it is impossible for you to prove that your
contention is that way.
Howard Ferstler
Howard Ferstler
>
> >>Mkuller wrote:>>
> >> However, I am absolutely convinced that if I were to listen to the amps
> >blind,
> >> I would not be able to identilfy which one I was listening to.
> >Howard Ferstler wrote:>
> >This tells us quite a bit. Obviously, your sighted
> >comparisons involved your feelings about each amp - feelings
> >that were unrelated to how they actually sounded.
> Sorry Howard, you have no way to know if my feelings were involved or if I
> based my conclusions soley on audible differences. Are you a mind-reader?
No I am not. The point is that you have no way of knowing,
either. Unless you do the comparison blind (at the same,
slow relaxed rate of speed that you did it at sighted) there
is no way for you to prove to me, anyone else, or even
yourself that what you think you heard was actually being
heard. You have no way to separate your subconscious
prejudices from the results unless you do the face off
blind.
> As an experienced listener, with no preconceptions, I believe I can put any
> biases aside.
Perhaps you can. However, there is only one way to validate
your claims for sure: do the comparisons blind.
> In fact my listening results were not at all what I would have
> predicted. I ended up preferring my old tube amps to a new, highly regarded
> (and nearly identical sounding) solid state model.
There is no fathoming the subconscious mind. That is why
blind comparisons are important if hair-splitting details
are important to you or you are trying to prove claims to
outsiders or do genuine research on the topic.
> >> The standard
> >> "objectivist" line is that if you can't identify the device blind, it
> >doesn't
> >> sound different from the other, or at least not different enough to matter.
> >That is my way of looking at it precisely. What else would
> >anyone want?
> Most audiophiles want to base their decisions on ALL the differences they hear.
And the key word is "hear." Doing it any other way (with the
assistance of the eyes, for instance) has the potential to
corrupt the results. Perhaps you have an iron will, but
until you check out its abilities with a blind test as a
reference standard we (and you) will never know for sure.
> If some of those subtle differences disappear or are not able to be recalled
> during the blind listening decision-making-process, then this procedure is not
> valid for making decisions about audio equipment using music.
I believe just the opposite. Just because it does not
validate your preconceptions does not make it invalid.
Indeed, what it does in that case is make your
preconceptions invalid.
> >> From my experience with these two amplifiers, that is clearly not the case.
> >> The subtle differences DO matter.
> >But you just confessed that you might not be able to tell
> >them apart during a blind comparison. Any subtle differences
> >you think were there during your sighted evaluations have
> >the potential to basically be no more than fabrications
> >based upon how you thought each amp should sound.
> I'll give you that is a "potential". An experienced listener (certainly an
> audio reviewer not investing his own money) should be able to separate the two.
Perhaps you should do the comparing blind, just to see if
what you say about the results will be that way. Why
speculate when doing the comparison blind is no big deal?
You seem to be vitally interested in this topic, and so it
would pay for you do do some genuine ABX comparisons to see
just what it is you are dealing with.
> >You have
> >no way to prove to anyone, including yourself, if the
> >differences you think you heard were really audible.
> So it's proof you're after. How can you prove those subtle differences you
> hear in sighted listening that cannot be identified or recalled from your
> memory during the blind "decision-making process" aren't really there, and not
> obscured by how your brain processes information?
This is speculating about what the brain is doing. You are
making a rather simple procedure (comparing stuff) into
rocket science. If there are audible differences, those
differences should show up in a comparison procedure that
involves just two things: the ears and the brain. Note that
I am not leaving the brain out as you seem to believe
happens with a blind comparison. Rather, I am eliminating a
variable - knowledge of what is playing - that will possibly
confuse the brain when it comes to what the ears are sending
to it in the way of rather straightforward information.
> >> So then why wouldn't I be able to tell the
> >> amps apart blind?
> >Because they they sounded the same.
> No they had subtle, but important audible differences.
How would you know unless you validate said claims by
comparing blind? Otherwise, you have no way of knowing
whether what you were supposedly were hearing was really
audible.
Also, just what is a "subtle but important audible
difference?" It strikes me that by definition a subtle
difference would not be particularly important.
Note that this question is just an aside, and has little to
do with the main topic at hand. It was not designed to start
a diversionary thread that would slide us past the important
things we are discussing. I was just curious.
> >> I suspect this has to do with how the brain processes information (so
> >please
> >> spare me the "you're imagining it" posts). As long as I am relaxed
> >listening
> >> to music, I can easily identify the subtle differences that separate these
> >two
> >> products. Once I begin to "problem-solve" to determine which amp I am
> >> listening to, my brain switches gears. No longer am I relaxed, but I begin
> >the
> >> "decision-making" process to determine which amp I it is. I begin to
> >search
> >> for clues, isolating one part of the sound and then another, to see which
> >> "amplifier-memory" it corresponds to.
> >You could still do this blind. Why does this laid-back test
> >of yours have to be done sighted? Do the same thing without
> >knowing which amp is playing. Do them in the same relaxed
> >manner, and see if the differences show up.
> From my experience, I don't believe you can "listen in a relaxed manner" so
> that you are aware of subtle details and make a decision about whether X is A
> or B at the same time. The matching/identifying/choosing process becomes the
> priority - not the relaxed listening. They are incompatible when the audible
> cues are subtle.
Again, if you are comparing in any way at all sighted I can
see no way why doing the same thing blind cannot be done in
just as relaxed manner.
Indeed, you were obviously listening for subtle details and
cues when you compared sighted (remember your comments above
about the "subtle but important audible differences" you
heard when comparing sighted), so why not do the same thing
with a blind protocol?
> >> As my brain changes gears here, I do not
> >> hear the subtle sounds at all, because I am too busy trying to focus on
> >> "differences" that I can identify, not the music as a whole (this may
> >explain
> >> why pink noise works better in these tests in some cases). I believe this
> >is
> >> why it is much easier to distinguish two products blind with gross
> >differences
> >> such as speakers than it is with amps, CD players, cables, etc. which tend
> >to
> >> have more subtle differences (if any).
> >No matter what you think about the effectiveness of sighted
> >comparisons, there is no way to tell if you are really
> >hearing differences unless you do the comparison blind. You
> >have no way of solidly knowing if you are fooling yourself
> >during sighted comparisons.
> And based on your comments, you have no way of knowing if you are losing subtle
> audible information during your blind tests and coming to incorrect conclusions
> based on them.
How would you lose subtle audible information during a blind
comparison? The only proof you can offer for this happening
is the fact that you think that subtle differences you heard
sighted would not be audible during a blind comparison. That
is not exactly proof. It mainly involves hope.
Basically, you are saying that if a blind test does not
validate your preconceptions the test must have something
wrong with it. You are setting yourself up as a reference
standard, and then saying that no protocol, however
rigorous, will be able to overcome your feelings about how
something should sound.
> >> Furthermore
> >> you should make your decisions based on those differences, not whether or
> >not
> >> the differences are evident in blind listening. Trust your ears.
> >Which is why the comparisons should be done blind if one
> >REALLY wants to know if differences are audible.
> Differences that are audible sighted are not always audible blind. Agreed?
No. You have no proof of that, other than the fact that a
blind comparison may not validate your ideas about how
something should sound. While you are obviously a reference
standard when it comes to personal taste, it is fatuous to
think that your status as a personal reference applies to
the real, objective world. And even on a personal level you
have no way to know for sure if you are indeed a reference.
You have a solipsistic situation where you claim to be
correct because you are correct.
> Now your explanation is preconception and bias and I reject that. I say it's
> due to the way the brain functions.
This is speculation. Basically, you are setting yourself up
as a reference standard that cannot be proven wrong by any
test that eliminates variables that you use to prove to
yourself that you are indeed a reference standard.
> >Your procedure may make you feel better and it may make you
> >a happier audio enthusiast to do things that way. However,
> >you will never know for sure if what you think you are
> >hearing is really audible if you do not do the comparing
> >blind. Not doing them blind always leaves the question
> >hanging.
> Doing them blind always leaves the question "What am I missing here? Why can't
> I identify it now when the differences were so obvious sighted?"
Obviously, you seem to think that introducing prior
knowledge about what is playing will help you pinpoint
audible differences that a lack of that kind of knowledge
will undermine.
I need to again point out that with an ABX test you can
compare A and B as much as you like, and that under those
conditions A and B will be known to you. You can do that
part of the test indefinitely and use that part of it to
pinpoint differences that you think are audible.
You can then switch to "X" and in the same relaxed manner
see if it has any characteristics of either A or B. I cannot
see how a comparison could be more straightforward or
honest.
Give it a try and see what you think. Until you actually do
the procedure you are basically guessing about its
inadequacies. Yes, I know the boxes are maybe a bit
expensive. However, you seem to be vitally interested in
this issue, and so one would think you would do what you
could to temporarily round up a device and gain some
first-person experience with it.
Howard Ferstler
Gary Eickmeier
> Gary, I know this goes against all of the hard-line objectivist dogma, but try
> to view it with an open mind. Listening to A and B is one thing, trying to
> decide which X is, is altogether a different activity which requires use of a
> different brain process. Call it "problem-solving" or "making a choice",
> whatever, but it is very different from listening and noting differences. To
> decide which choice X is, you must match its audible cues to the ones in your
> memory, identify it as A or B and make a conscious choice. This requires
> higher level brain activity than merely listening. This higher level brain
> activity interferes with recalling subtle audible cues.
To decide whether B is different from A, you must match its audible
clues to ones in your memory, decide whether it is the same or not,
and make a conscious choice. This requires higher level brain activity
than merely listening. This higher level brain activity interferes
with recalling subtle clues.
How do you manage it?
> So in your view, you listen to A and B and hear easily identifiable, but subtle
> cues which distinguish them, then switch to X and can't identify it accurately.
> What is the explanation?
That you couldn't hear any real differences. What is your explanation?
Gary Eickmeier
Howard Ferstler
I simply cannot see how this is the case. You are listening
for differences that you claim are there with the sighted
A/B part of the test. What on earth would change?
Basically, you are saying that an ABX test somehow
introduces confusing issues into the mix that cloud the
ability of the brain to spot "real" differences. This is
speculation, and the only proof you have that this happens
is the fact that a blind comparison will not always validate
what you think you hear when you do a sighted comparison.
You are essentially using a test procedure that has
inherent, potential flaws as a reference for one that has
flaws you claim exist, because the first test is postulated,
and only postulated, as a reference standard. You are
essentially saying that a sighted test tells the truth
because a sighted test tells the truth. If a blind test
contradicts that, a blind test must be, ipso facto, a bad
test.
> Call it "problem-solving" or "making a choice",
> whatever, but it is very different from listening and noting differences.
Nonsense. You can still do a listening test that involves
noting differences with the names of the items being
listened to being unknown.
> To
> decide which choice X is, you must match its audible cues to the ones in your
> memory, identify it as A or B and make a conscious choice.
Well, when comparing A and B during an ABX test, or
comparing by the method you noted previously, you still have
to match audible cues by memory and make a conscious choice
as to whether differences are audible.
You assume that doing that blind somehow clouds the issue,
but the only thing you have to offer up as proof that is
that it does not jive with your preconceptions. You set
yourself up as a reference standard, and then discount any
outside procedure that challenges that reference.
> This requires
> higher level brain activity than merely listening. This higher level brain
> activity interferes with recalling subtle audible cues.
This is nothing more than speculation. Using a sighted
technique, you set yourself up as a reference standard and
then decry any blind protocol that will not let your
prejudices be validated.
> So in your view, you listen to A and B and hear easily identifiable, but subtle
> cues which distinguish them, then switch to X and can't identify it accurately.
> What is the explanation?
My explanation is that when comparing sighted the listener's
preconceptions were interfering with the procedure.
Switching to the blind protocol removed the influence of
those preconceptions and the listener had to actually
"listen" for differences.
Howard Ferstler
Arny Krueger
> Gary, I know this goes against all of the hard-line objectivist
> dogma, but try to view it with an open mind. Listening to A and B
> is one thing, trying to decide which X is, is altogether a
> different activity which requires use of a different brain
> process. Call it "problem-solving" or "making a choice",
> whatever, but it is very different from listening and noting
> differences. To decide which choice X is, you must match its
> audible cues to the ones in your memory, identify it as A or B
> and make a conscious choice. This requires higher level brain
> activity than merely listening. This higher level brain activity
> interferes with recalling subtle audible cues.
I don't know how many times I have to post this, but this is
something that can be stipulated to be exactly the case without
affecting the relevance of blind tests because it affects any kind of
evaluation process, whether sighted or blind.
> So in your view, you listen to A and B and hear easily
> identifiable, but subtle cues which distinguish them, then switch
> to X and can't identify it accurately. What is the explanation?
Under those circumstances I know that I wasn't hearing what I thought
I heard. Happens to just about everybody quite often.
Howard Ferstler
>
> >Stewart Pinkerton wrote:>
> >OK, that's another claim based on sighted 'listening'. Note please
> >that you *are* performing a conscious comparison here, so let's not
> >have any bull about how this is not how you normally listen to music.
> >You have clearly stated that you are playing the same pieces of music
> >on each amp, matching levels, and listening for differences.
> Wrong. When I'm listening to music in a relaxed fashion or listening
> critically and taking notes, those are similar activities which I suspect
> utilize the same brain functions. It's when one switches activities from
> listening to making a decision which requires "matching/identifiing/choosing"
> that things change. Somehow this function seems to interfere with recalling
> memories of subtler audible cues.
This is a pretty radical claim. What can you offer up as
evidence for this? And, no, your feelings about how you can
hear differences sighted is not evidence. You cannot set
yourself up as a reference standard if you are trying to
prove a point that exists outside of your own existential
experiences. You may satisfy yourself with your claims, but
there is no way to prove it to others.
This is in stark contrast to an ABX test that will at least
be able to prove to others - an to yourself - that you
cannot hear differences between some components when
comparing blind.
> More often than not when the audible cues
> are subtle,
> blind listening results in "no difference heard". This is opposed to sighted
> listening (no decision-making is required) where subtle audible differences
> are easily heard.
How would you know this is true? You know what is playing,
and so you could easily be fooled by your own
preconceptions. It is fatuous to claim that because
differences disappear when listening blind those differences
must somehow be obscured by the blind-comparison protocol.
Most people would say just the opposite: sighted comparisons
introduce potential prejudices that blind comparisons will
not allow to be introduced.
> >> Once I begin to "problem-solve" to determine which amp I am
> >>listening to, my brain switches gears. No longer am I relaxed, but I begin
> >the
> >>"decision-making" process to determine which amp I it is. I begin to search
> >>for clues, isolating one part of the sound and then another, to see which
> >>"amplifier-memory" it corresponds to.
> >And this would be different in a blind test, how? You have already
> >admitted to careful level matching and listening to the same pieces of
> >music on each amp, so what's the problem with doing *exactly* the same
> >thing, with the sole difference that you don't *know* which amp is
> >playing.
> Because this requires an activity which is different from "listening" and
> casually comparing. It requires making a decision -
> matching/identifiing/choosing.
But you also made a decision (matching/identifying/choosing)
when you did your sighted comparisons. You felt that one amp
sounded better than the other. You admitted that previously.
Both your sighed comparison and a blind comparison work the
same way in that respect.
Howard Ferstler
Steve Tew
news:ajm1e6$k4t$1...@bourbaki.localdomain...
Right, state of mind does make a difference. ABX testing is flawed
in the same sense that any test is flawed. The test environment
generates anxiety in the testee (evaluator) which modifies his or her
performance. It certainly doesn't render it without merit, but
translation into a data set which would optimally yield a meaningful
decision - making aid to anyone but the evaluator is questionable,
unless it is known that the personal taste of the evaluator parallels
that or the data user, and there is a prior familiarity with at least
one of the equipments subjected to evaluation. ABX testing is a test
of subjective human performance, as well as an objective equipment
comparison. What is "real"?
"There is no spoon..."
Steve
BEAR
No, not "any test." There are many tests that are pretty darn valid,
without going completely metaphysical about everything. What we're
talking here is a test that involves human perception to generate an
outcome.
You go on to pretty much say this - but it's not "any test."
> The test environment
> generates anxiety in the testee (evaluator) which modifies his or her
> performance. It certainly doesn't render it without merit, but
> translation into a data set which would optimally yield a meaningful
> decision - making aid to anyone but the evaluator is questionable,
> unless it is known that the personal taste of the evaluator parallels
> that or the data user, and there is a prior familiarity with at least
> one of the equipments subjected to evaluation. ABX testing is a test
> of subjective human performance, as well as an objective equipment
> comparison. What is "real"?
>
> "There is no spoon..."
>
> Steve
--
Howard Ferstler
> Right, state of mind does make a difference. ABX testing is flawed
> in the same sense that any test is flawed. The test environment
> generates anxiety in the testee (evaluator) which modifies his or her
> performance.
So what? Why should a blind test engender any more anxiety than a
sighted test, other than the fact that during a blind test the
participant cannot cheat. He has to put up or shut up. (Well, of
course, when some guys flunk they do anything but shut up.) I see no
bigger deal with a blind test than with a sighted test, but obviously
those who are sweating the status of their golden ears or the
performance of a favorite set of wires or amp will probably have a
rather unsettled state of mind during the test. Life is tough.
> It certainly doesn't render it without merit.....
It certainly should have more merit than a sighted test, because there
is no way to cheat.
Howard Ferstler
Mark Ovchain
Oh, Mike, Mike, Mike!
> My overall conclusion from this experience is that if you hear differences
> between components in casual listening, those differences are REAL. > Furthermore
> you should make your decisions based on those differences, not whether or not
> the differences are evident in blind listening. Trust your ears.
I quite frankly don't know that many people disagree with you, if
you're talking about establishing your own preferences, for your own
purposes, but why, oh WHY must you invent reams of myth about why AB
testing is bad, instead of simply accepting the seemingly obvious
fact that your whole-sense (somebody called it gestalt, I think)
perception might rely on more than mere sound?
There's nothing wrong with that, you know.
Your entire experience is entirely typical, and entirely within the
realm of the outcome from cross-sensory coupling, inadvertant
self-influence, and the well-established facts about overdetection of
audible differences.
That's fine, you know. You found something you like. Not only is
there no harm in sight and other senses providing some decision
input, it would seem sensible, after all, in most cases, you do have
to look, smell, and otherwise sense the amplifier, cables, and
what-have-you, as well as hear the final result of it's intended
outputs, and including those experiences would only seem rational.
Is there some reason you won't simply accept that, and instead you
continue to spout ill-formed, un-demonstrated, untestable hypothesis
about well-established testing methods, while not accepting at all,
it would appear, the known results of cross-sensory effects and the
like?
ludovic mirabel
· The evidence so far is all in favour of Mike's hypothesis that
trying to guess X is a problem for many. For instance ( and only one
instance to keep it short) in Harman-Kardon "listening room" website
· Sean Olive reports individual ABX accuracy of his trained and
retrained panel, composed one would assume of H-K audio professionals,
as ranging between 30% and 80%. Your unspoken assumption, frequently
voiced by others as well, is that if they do't here it ABXing they
only "imagine" it when they hear it sighted" .
· This assumes that some of the H-K audio professionals just don't
hear what the other ones do. You don't even consider the other
possibility that some people just are bad ABX performers. Even after
training! S. Olive retrained his panelists- some improved-some didn't.
Again you'd say that they got trained in their music perceptions while
I'd say that it is just as likely that some got better at ABXing.
Neither I nor you can prove which ones did what and how much of it.
But I don't claim that unless you hear it ABXing you don't hear
it,period. I don't have to prove anything . You do.
· There is no experimental evidence for your hypothesis'.To you it
may seem self-evident, But you must know that the history of science
is littered with "self-evident truths". Till you can prove your
belief it is only a bias. My and Mike's well-grounded belief is that
for many ABXing is a hurdle to a varying, individual degree when it
comes to recognising differences in musical reproduction by electronic
components . Where I differ from Mike is that I'm not axiomatically
certain that some of the ABX incapables will not perform adequately
using other blind protocol. But I can't prove that either.
· Ludovic Mirabel
Gary Eickmeier <geic...@tampabay.rr.com> wrote in message news:<ajhors$ce1$1...@bourbaki.localdomain>...
> Mkuller wrote:
>
> > I suspect this has to do with how the brain processes information (so please
> > spare me the "you're imagining it" posts). As long as I am relaxed listening
> > to music, I can easily identify the subtle differences that separate these two
> > products. Once I begin to "problem-solve" to determine which amp I am
> > listening to, my brain switches gears.
>
> The oft repeated fallacy here is that in blind testing you must not
> know which amp is which, or you cannot see the two devices. Au
> contraire, you can set them up right next to each other, with a bright
> spotlight on the appropriate one that activates when you switch to A
> or B, listen to your heart's content for all the subtle differences
> between them, all the while knowing precisely which you are listening
> to and when.
>
> Then, when you have "locked on" to some particular difference that you
> know you can use to identify them blind, you switch between A and X,
> then B and X, and determine which one X sounds like. If the difference
> you heard sighted was real, it should be easy to also hear it blind.
> Notice that you can still see both amps on the table or floor; you
> just don't get the spotlight telling you that you are listening to A
> when X is playing. The ONLY difference is that you are deprived of the
> prior knowledge of which is playing when you select X, so you must
> decide based on sound alone. Which is, of course, exactly the test we
> are after.
>
> Beautiful, isn't it?
>
> Gary Eickmeier
Mark Ovchain
> No, not "any test." There are many tests that are pretty darn valid,
> without going completely metaphysical about everything. What we're
> talking here is a test that involves human perception to generate an
> outcome.
My, my, my. What a stunning example of proof by blatant assertion.
Well, laddie, that's your position, and you get to have it, but now
you're going to have go supply us all with some hard, concrete,
testable, falsifible evidence to support your claim.
It's time for you to show that any test hasn't the one and only
problem that an ABX test (using the definition of an A/B fixed
presentation of reference and alternate reference/probe, and X as a
probe condition that is one or the other of A or B, where the listener
gets to control where and when one switches, when one loops music,
etc, as it is presently used in the art, please, no straw men or
malicious miscomprehension, please) has, that being that the subject
knows that he or she eventually must provide the test with an answer.
That's something that's true of everything, including a purely
subjective review, so I think you're going to have to back off your
blatant assertion and admit that you're without a doubt completely,
totally, and hideously wrong.
What I'm puzzled by is your insistance on this. Why must you do this?
normanstrong
news:ak0f1...@enews2.newsguy.com...
The only way to eliminate "test anxiety" is to run the test without
the knowledge of the testee, who is therefore relaxed and under no
stress. Unfortunately, the people who object to DBT, also consider
"ambush" testing unacceptable--even immoral. It's a no-win
situation; either the subject is not concentrating because he doesn't
know it's a test, or he does know, and test anxiety is the excuse.
Norm Strong
Norman M. Schwartz
A/B/Whatever testing, and a statistical analysis be required to
proove it exists, might I please ask why bother at all?
BEAR
You guys seem to insist upon focusing on everything except for what I
said originally... so I ask, why bother at all? (as far as this sub
thread is concerned)
What I spoke of is how ABX/DBT or any other test is not *so simple*
because of the way that the brain processes sound and makes sense of
it.
To that extent it is possible to have a completely "valid" ABX/DBT
test that shows very little about what *can* be heard. By the same
token it is possible to have a valid ABX/DBT test that *does* show
quite a bit about what *can* be heard.
Personally, I have no problem with any technique that brings the
state of the art to a higher level. This includes better measurement
techniques and equipment as well as various "testing" and
"auditioning" methods.
_-_-bear
Emerson Wood
> Mkuller wrote:
> >
> > >Stewart Pinkerton wrote:>
> > >OK, that's another claim based on sighted 'listening'. Note please
> > >that you *are* performing a conscious comparison here, so let's not
> > >have any bull about how this is not how you normally listen to music.
> > >You have clearly stated that you are playing the same pieces of music
> > >on each amp, matching levels, and listening for differences.
>
> > Wrong. When I'm listening to music in a relaxed fashion or listening
> > critically and taking notes, those are similar activities which I suspect
> > utilize the same brain functions. It's when one switches activities from
> > listening to making a decision which requires "matching/identifiing/choosing"
> > that things change. Somehow this function seems to interfere with recalling
> > memories of subtler audible cues.
>
> This is a pretty radical claim. What can you offer up as
> evidence for this? And, no, your feelings about how you can
> hear differences sighted is not evidence. You cannot set
> yourself up as a reference standard if you are trying to
> prove a point that exists outside of your own existential
> experiences. You may satisfy yourself with your claims, but
> there is no way to prove it to others.
Howard,
An unstated premise behind all of your reasoning seems to
be, "Truth about human perception is objectively knowable
or provable." That is, it can be ascertained through repeatable
experiments that can be verified by independent observers.
You keep saying, "There's no way to prove your point" as
if that stopped the argument right there.
In my experience, attempts to make objective knowledge
about human experience have all succumbed to illusion.
If you think about it, the universe has given us no promise
that human experience, or how human experience interacts with
objects, should be objectively knowable. That's simply an
assumption of those people who cling to objective knowledge
as the only "real" knowledge.
But, a few thought experiments can easily show that
bias or distortion in the observer is perfectly capable
of creating self-consistent, flawless "realities." I'm
sure you agree...after all, you think that sighted bias
is capable of creating the consistent illusion that
brand A always
sounds better than brand B, even when it so happens that
brand A and brand B are the same brand packaged in
differently-colored boxes. If a subjectivist told you,
"But brand A *always* sounds better than brand B," you'd
say, "Sure. That's perfectly consistent with sighted
bias."
DBT is a kind of lens, a way of viewing the world. Is it
illusion-free? I don't think so. It uses obvious mechanisms
that are begging for the creation of illusion, *as it is commonly
done*. (This is not a discussion about "DBT" as the galaxy-sized
concept of "everything that could be DBT," this is a discussion
about how it's actually done in the bulk of experiments that
form the body of objective knowledge about audio.)
A human observes. Is this human trying to answer a question?
What question? Or is this human *not* trying to answer
a question? This very much impacts what details are
perceivable. Human experience is *not at all* like standing
in a room where every object is visible, clearly labelled,
and clearly categorized, and any question about the contents
of that room can be answered with a straightforward, logical
process. So that's an obvious distortion mechanism in DBT;
DBT proponents seem to assume that you can pick something to
listen to, directly
turn your attention to it, and then be able to perceive any
relevant detail. That's just not how humans work. (Again,
this is talking about how the majority of humans try to
operate in a DBT, not about the cosmic distant limits of
how someone might theoretically function in a DBT.)
It's how some humans *try* to operate themselves, and the
result is illusion.
I realize this is a controversial perspective, but it is one
that can be discovered and verified through careful experimentation
with subjective perception. I can't prove it to you, but
you can't prove your perspective to me either.
Another aspect of DBT that is typically present: it takes
the focus away from broad time scales, listening to pieces
as a whole. This is true even if the test involves long
listening segments, because *typically* the listener is
mentally trying to ask and answer questions at a high
rate within the listening experience.
From my perspective, these are obvious illusion mechanisms
that are perfectly capable of creating verifiable, repeatable,
objective illusions. No objectivist on this group has
shown any sign of willingness to even begin considering these
illusion mechanisms---no doubt because they cling so tightly
to the security of believing that real knowledge is objective.
>
> This is in stark contrast to an ABX test that will at least
> be able to prove to others - an to yourself - that you
> cannot hear differences between some components when
> comparing blind.
It can prove that you can't come up with the right
answer, not in *any* blind situation, but in *that*
blind situation.
>
> > More often than not when the audible cues
> > are subtle,
> > blind listening results in "no difference heard". This is opposed to sighted
> > listening (no decision-making is required) where subtle audible differences
> > are easily heard.
>
> How would you know this is true? You know what is playing,
> and so you could easily be fooled by your own
> preconceptions. It is fatuous to claim that because
> differences disappear when listening blind those differences
> must somehow be obscured by the blind-comparison protocol.
>
> Most people would say just the opposite: sighted comparisons
> introduce potential prejudices that blind comparisons will
> not allow to be introduced.
Right--blind comparisons just have their own prejudices.
>
> > >> Once I begin to "problem-solve" to determine which amp I am
> > >>listening to, my brain switches gears. No longer am I relaxed, but I begin
> the
> > >>"decision-making" process to determine which amp I it is. I begin to search
> > >>for clues, isolating one part of the sound and then another, to see which
> > >>"amplifier-memory" it corresponds to.
>
> > >And this would be different in a blind test, how? You have already
> > >admitted to careful level matching and listening to the same pieces of
> > >music on each amp, so what's the problem with doing *exactly* the same
> > >thing, with the sole difference that you don't *know* which amp is
> > >playing.
>
> > Because this requires an activity which is different from "listening" and
> > casually comparing. It requires making a decision -
> > matching/identifiing/choosing.
>
> But you also made a decision (matching/identifying/choosing)
> when you did your sighted comparisons. You felt that one amp
> sounded better than the other. You admitted that previously.
> Both your sighed comparison and a blind comparison work the
> same way in that respect.
>
> Howard Ferstler
I don't know about Mike, but I think that most kinds
of quick switching, sighted or blind, are subject to
the illusions following from narrowed focus of attention.
-Emerson
Steve Tew
environment for the evaluator in a DBT than in a situation where
equipment is swapped and listened to casually. There is the pressure
to identify, rather than just enjoy and compare. It may seem
sophomoric and unscientific, but that's O.K. Both methods have their
place and value. It's just that DBT is not as purely objective as
we would like. In the case of a panel DBT where a notable majority
found that there was a better sound stage presentation in one of a
few preamps tested, I think that is of more value than a single
listener doing a swap, but not of more value than a large body of
listeners who had reached that conclusion with casual listening.
In the case of the same panel finding no statistally significant
difference between types of speaker cables, and a comparative number
of anecdotal accounts from casual listeners who claim differences, I
would just have to use that as a data that could steer me to doing my
own evaluation. One reason is that I have not, in my limited
experience, been introduced to a DBT setup for speaker cables that
emulates a realistic connection, as would exist in the home.
Still, there is no spoon...
Steve
"Howard Ferstler" <fer...@attglobal.net> wrote in message
news:ak0f1...@enews2.newsguy.com...
S888Wheel
>> in the same sense that any test is flawed. The test environment
>> generates anxiety in the testee (evaluator) which modifies his or her
>> performance.
>
>
>So what? Why should a blind test engender any more anxiety than a
>sighted test, other than the fact that during a blind test the
>participant cannot cheat. He has to put up or shut up. (Well, of
>course, when some guys flunk they do anything but shut up.
> I see no
>bigger deal with a blind test than with a sighted test, but obviously
>those who are sweating the status of their golden ears or the
>performance of a favorite set of wires or amp will probably have a
>rather unsettled state of mind during the test. Life is tough.
This reminds me of people who wonder why people with Terets simply
just don't do the things they do. The human mind is a complex system.
It is a well known fact that seemingly related tasks such as writing
language and speaking it use very different parts of the brain. You
can't just will it away. If the task of doing ABX DBTs is masking
something in our perceptions it is a problem. That this is a problem
with ABX listening tests hasn't been proven but has it been
disproven? So any assumptions either way are just that, assumptions.
Unless there have been some tests I don't know about that addressed
this issue. It would be very easy to test whether or not the brain
changes gears in ABX DBTs although I suspect it would be expensive.
Tests of brain activity while different tasks are performed is
nothing new and is very scientific. You may not see a bigger deal
with a blind test than with a sighted test but can you claim that
this in and of itself has been properly tested? Unfortunately time
and money make the exploration of this probably unlikely but it would
be interesting to map brain activity during ABX tests and during
sighted tests to see if there were a substantial difference and if
there were, to try to devise a proper controled listening test that
did not change the brain activity of the task of making sighted
comparisons.
Stewart Pinkerton
wrote:
>If a supposed sonic difference is so subtle that "golden" ears,
>A/B/Whatever testing, and a statistical analysis be required to
>proove it exists, might I please ask why bother at all?
Because that's what the high end is supposed to be all about -
reaching from the very good to the almost perfect.
Of course, if you read the audio ragazines, you may well believe that
$1,000 a foot speaker cables simply 'blow away' mere $300 a foot
cables! Back here in the real world, they both sound just the same as
$1 a foot Rat Shack wire....................
Gary Eickmeier
something so irrelevant.
Why all the interest in blind testing? Has someone been pushing it for
audiophiles? Why does this topic keep coming up so often here?
Gary Eickmeier
Mkuller wrote:
>
> I recently purchased a new amplifier and have spent a lot of time listening to
> it and comparing it to my old amp. In this case, the two amplifiers sound much
> more similar than I would have expected, but I hear small differences between
> them that I consider important, especially for long term listening pleasure.
> Now when I sit and listen to music I relax and enjoy the experience. When I
> decide it's time to switch them, I listen to 3 or 4 cuts that demonstrate
> specific musical qualities and take notes. After careful level matching (with
> a meter), I switch the amps and listen to the same music in the reverse order
> and take notes again. As I said, I have consistantly noted important subtle
> differences in comparing the sound of the two amps playing music. One of the
> two amps would not satisfy me as much long term, particularly after having
> listened at length to the other one.
>
> However, I am absolutely convinced that if I were to listen to the amps blind,
> I would not be able to identilfy which one I was listening to. The standard
> "objectivist" line is that if you can't identify the device blind, it doesn't
> sound different from the other, or at least not different enough to matter.
> From my experience with these two amplifiers, that is clearly not the case.
> The subtle differences DO matter. So then why wouldn't I be able to tell the
> amps apart blind?
>
> I suspect this has to do with how the brain processes information (so please
> spare me the "you're imagining it" posts). As long as I am relaxed listening
> to music, I can easily identify the subtle differences that separate these two
> products. Once I begin to "problem-solve" to determine which amp I am
> listening to, my brain switches gears. No longer am I relaxed, but I begin the
> "decision-making" process to determine which amp I it is. I begin to search
> for clues, isolating one part of the sound and then another, to see which
> "amplifier-memory" it corresponds to. As my brain changes gears here, I do not
> hear the subtle sounds at all, because I am too busy trying to focus on
> "differences" that I can identify, not the music as a whole (this may explain
> why pink noise works better in these tests in some cases). I believe this is
> why it is much easier to distinguish two products blind with gross differences
> such as speakers than it is with amps, CD players, cables, etc. which tend to
> have more subtle differences (if any).
>
> My overall conclusion from this experience is that if you hear differences
> between components in casual listening, those differences are REAL. Furthermore
> you should make your decisions based on those differences, not whether or not
> the differences are evident in blind listening. Trust your ears.
> Regards,
> Mike
Howard Ferstler
> If a supposed sonic difference is so subtle that "golden" ears,
> A/B/Whatever testing, and a statistical analysis be required to
> proove it exists, might I please ask why bother at all?
This has been one of my points for some time. The fact is that
differences that are claimed to be like "night and day" during sighted
comparisons (particularly at dealer demos, where the amplifier-gain
factor might be stacked in favor of the products the clerk wants to
sell the most) often become maddeningly subtle when tests with the
same two components are done level matched and blind.
However, rather than seeing the situation as a revelation or at least
acquiescing in the fact that maybe things were being blown all out of
proportion during the sighted comparisons, many people will go on and
on and on about statistical hair splitting details that have nothing
to do with the fact that products that seemed to sound radically
different during sighed comparisons (with the levels almost certainly
not carefully matched) suddenly become tiny or impossible to hear at
all during blind, level-matched, quick-switch comparisons.
To me, that is a very critical factor that it regularly glossed over
in favor of hair splitting details about statistical analysis or
debates about how blind comparisons, with quick switching, somehow
cloud the perceptive faculties to the extent that gross differences
somehow disappear.
Howard Ferstler
Nousaine
>
>>> Right, state of mind does make a difference. ABX testing is flawed
>>> in the same sense that any test is flawed. The test environment
>>> generates anxiety in the testee (evaluator) which modifies his or her
>>> performance.
>>
>
>>
>>So what? Why should a blind test engender any more anxiety than a
>>sighted test, other than the fact that during a blind test the
>>participant cannot cheat. He has to put up or shut up. (Well, of
>>course, when some guys flunk they do anything but shut up.
>
>> I see no
>>bigger deal with a blind test than with a sighted test, but obviously
>>those who are sweating the status of their golden ears or the
>>performance of a favorite set of wires or amp will probably have a
>>rather unsettled state of mind during the test. Life is tough.
>
>This reminds me of people who wonder why people with Terets simply
>just don't do the things they do. The human mind is a complex system.
>It is a well known fact that seemingly related tasks such as writing
>language and speaking it use very different parts of the brain. You
>can't just will it away.
This is precisely the problem with open evaluation for sound quality
differences. There are human response bias mechanisms (level/quality
confusion, pro-choice decision tendency are two) that cannot be
willed away.
>If the task of doing ABX DBTs is masking
>something in our perceptions it is a problem. That this is a problem
>with ABX listening tests hasn't been proven but has it been
>disproven?
There is no evidence that controls for human listening response bias
affect anything except non-sonic bias mechanisms.
>So any assumptions either way are just that, assumptions.
>Unless there have been some tests I don't know about that addressed
>this issue. It would be very easy to test whether or not the brain
>changes gears in ABX DBTs although I suspect it would be expensive.
Even if brain activity were found to change that is not evidence that
any sonic factors would have been masked.
random
questionable people's evaluation of fuzzy data is and how easily they
can convince themselves of phenomenon that do not exist. A friend of
mine who was involved in the development of one of the early video
game systems decided to replace the regular game controller with a
Thought Activated controller. This consisted of a headband with
electrodes similar to an EEG setup going into a processing box and
then hooking into the video game in place of the game controller. By
putting on the headband and thinking either RIGHT or LEFT you could
control the game to move either right or left. He admitted to people
that it was far from perfect yet and you had to think very clearly to
get it to work.
Of the several dozen people exposed to this device almost every
person was utterly convinced that he was having some control effect
and that they were slowly getting better at controlling the game the
more they used it. They were mostly very impressed and enthusiastic
over the promise of this thought control system.
Only later did my friend tell them that it was a hoax and that the
processing box merely had a random number generator in it controlling
the game controller input.
People are highly influenced by preconceptions and by pre exposure to
contaminating data. Another example of this is crime witnesses. Only
the first time they are shown photographs for identification is the
information found to be useful. Any subsequent photo interrogation
after the initial questioning and the information becomes essentially
worthless.
bk
Jonathan Babiak wrote:
> ps All typographical errors become the property of the finder. No
> refunds.
Howard Ferstler
> >> Right, state of mind does make a difference. ABX testing is flawed
> >> in the same sense that any test is flawed. The test environment
> >> generates anxiety in the testee (evaluator) which modifies his or her
> >> performance.
> >So what? Why should a blind test engender any more anxiety than a
> >sighted test, other than the fact that during a blind test the
> >participant cannot cheat. He has to put up or shut up. (Well, of
> >course, when some guys flunk they do anything but shut up.
>
> > I see no
> >bigger deal with a blind test than with a sighted test, but obviously
> >those who are sweating the status of their golden ears or the
> >performance of a favorite set of wires or amp will probably have a
> >rather unsettled state of mind during the test. Life is tough.
> This reminds me of people who wonder why people with Terets simply
> just don't do the things they do. The human mind is a complex system.
Yes it is. However, I do not see how this relates to one's ability to
hear differences or not hear differences during a blind comparison.
> It is a well known fact that seemingly related tasks such as writing
> language and speaking it use very different parts of the brain. You
> can't just will it away. If the task of doing ABX DBTs is masking
> something in our perceptions it is a problem. That this is a problem
> with ABX listening tests hasn't been proven but has it been
> disproven?
I just do not see it as a big deal. The problem with your observation
is that there is no way to validate the claims of those doing sighted
tests. They may not even be aware of their own prejudices when they
pick out a favorite, or impressive-looking, or very expensive amp as
sounding better than another, less openly impressive appearing
competing model.
Sure, you can say that the brain may be effected in a negative manner
if you cannot know what is playing during a comparison, but that is
essentially a guess. Basically, it says that "If a blind test does not
validate what I heard sighted then there must be something wrong with
the test - because I know what I heard during the sighted comparison.
I heard it, and if I cannot hear it during a DBT there is something
wrong with the DBT. It simply cannot be that there is something wrong
with me."
However, this is just speculation. There is nothing physiological that
I know of that says that a DBT will rob the brain of some of its
ability to evaluate sound. All a DBT does is eliminate a very
troublesome variable: the knowledge of what component is playing.
Howard Ferstler
Howard Ferstler
> > How would you know this is true? You know what is playing,
> > and so you could easily be fooled by your own
> > preconceptions. It is fatuous to claim that because
> > differences disappear when listening blind those differences
> > must somehow be obscured by the blind-comparison protocol.
> >
> > Most people would say just the opposite: sighted comparisons
> > introduce potential prejudices that blind comparisons will
> > not allow to be introduced.
> Right--blind comparisons just have their own prejudices.
And what might those be? The fact is that no matter what you might
think about blind comparisons screwing up one's perceptive
capabilities, at least the results cannot be fabricated. You either
can or cannot hear differences, and there is no way to pick out your
favorite amp or set of wires because they happen to be your favorite.
You have to judge by sound alone. You cannot be prejudiced one way or
the other, because you have no idea which component is playing during
the blind phase of the test.
This is in contrast to a sighted comparison of any kind, where there
is no way to know if the results of the test were because genuine
differences were audible or if the knowledge of what was playing
somehow colored the perceptions of the participant and caused him to
pick the product that he subconsciously preferred.
And there is no way the participant can prove to anyone else, or in my
opinion even to himself for absolutely sure, that he was actually
hearing differences during a sighted comparison that would not show up
audible differences during a DBT with the same components.
Sighted comparisons always result in the participant not ever being
absolutely sure of what he is hearing. He may feel good about the
results for a while, but down deep inside there will always be that
nagging doubt. Well, I suppose that is why upgrading is so common in
the high-end community.
Howard Ferstler
Howard Ferstler
> · Not so simple Mr. Eickman.
> · The evidence so far is all in favour of Mike's hypothesis that
> trying to guess X is a problem for many.
Well, actually, if there are audible differences one will not have to
"guess" X at all. You chose, based upon a genuine sonic difference.
You only have to "guess" if there are no audible differences.
Admittedly, that can cause frustrations.
> For instance ( and only one
> instance to keep it short) in Harman-Kardon "listening room" website
> · Sean Olive reports individual ABX accuracy of his trained and
> retrained panel, composed one would assume of H-K audio professionals,
> as ranging between 30% and 80%. Your unspoken assumption, frequently
> voiced by others as well, is that if they do't here it ABXing they
> only "imagine" it when they hear it sighted" .
> · This assumes that some of the H-K audio professionals just don't
> hear what the other ones do.
This most certainly makes it important for those who doubt the hearing
acuity of those individuals to do a bit of ABX testing of their own.
That way, they can determine for sure if audible differences really do
exist - for them.
> You don't even consider the other
> possibility that some people just are bad ABX performers.
Some people can hear better than others. No doubt. Of course, even
those people only have to satisfy a standard that pertains to them. If
you doubt their hearing acuity, do your own tests, with your own gear,
and see what comes up. If the results they come up with look
reasonably valid to you, stop worrying about the "sound" of amps and
wires, spend less money on said hardware, and use the savings to
purchase more recordings.
> Even after
> training! S. Olive retrained his panelists- some improved-some didn't.
Again, for consumers all that would matter is whether they could or
could not hear differences. The results of other individuals would be
irrelevant. Of course, some people read about such tests and leave it
at that. They assume that if most of those participating had at least
reasonably normal hearing and still had problems hearing differences
between stuff like amps and wires (even those who could if they
listened very closely) those differences are probably inconsequential.
This may seem like a short cut, but one of the reasons people bother
to publish the results of such tests is that they want to offer other
people a way to shortcut the issue and stop worrying about the sound
of amps and wires.
> Again you'd say that they got trained in their music perceptions while
> I'd say that it is just as likely that some got better at ABXing.
Good for them. However, that still does not invalidate the usefulness
of the procedure on a personal level. What matters is what they hear
or what you hear in such a comparison, be you, or they, sharp eared or
near deaf.
> Neither I nor you can prove which ones did what and how much of it.
> But I don't claim that unless you hear it ABXing you don't hear
> it, period. I don't have to prove anything . You do.
Good point, in a way. However, if you claim to hear differences
between decently built amps or wires during a sighted comparison I
hope you will allow me to doubt your claims.
> · There is no experimental evidence for your hypothesis'.To you it
> may seem self-evident, But you must know that the history of science
> is littered with "self-evident truths". Till you can prove your
> belief it is only a bias. My and Mike's well-grounded belief is that
> for many ABXing is a hurdle to a varying, individual degree when it
> comes to recognising differences in musical reproduction by electronic
> components .
The problem for both of you is that there is no way for you guys to
actually prove to others that this is the case when comparing sighted.
You certainly could not get a technical paper published with that kind
of research data.
Worse, there is no way for you to be sure of the results even for
yourselves. You may kind of feel confident about your hearing acuity
when comparing sighted, but down deep inside there may always be that
small doubt: "Did I really hear those differences?" I suppose this is
why some high-end enthusiasts upgrade so often. They are still looking
for some kind of rock-solid proof, but they are using a listening
technique that will never give them that kind of proof.
But if upgrading is your hobby, have at it.
Howard Ferstler
Mkuller
wrote:>
>Perhaps you can. However, there is only one way to validate
>your claims for sure: do the comparisons blind.
>
valid. Can you understand that?
>There is no fathoming the subconscious mind.
Sometimes the "conscious" mind is difficult to explain as well.
>That is why
>blind comparisons are important if hair-splitting details
>are important to you or you are trying to prove claims to
>outsiders or do genuine research on the topic.
>
What you call "hair-splitting detail" are what High End audio is all about.
With mid-fi I agree they aren't important. Remind me which it is you deal
with.
Obviously, you have accepted blind testing without question. Blind testing
will work fine when the differences are large. For subtle differences, blind
testing causes the differences to be obscured.
>> >You have
>> >no way to prove to anyone, including yourself, if the
>> >differences you think you heard were really audible.
>
>> So it's proof you're after. How can you prove those subtle differences you
>> hear in sighted listening that cannot be identified or recalled from your
>> memory during the blind "decision-making process" aren't really there, and
>not
>> obscured by how your brain processes information?
>
You are speculating that sight interferes with what you are hearing. It can
but it doesn't always. Blind testing appears to always obscure subtle audible
differences because of the "decision-making" process involved. Some blind
listeners describe trying to identify X as A or B as confusion or just stress.
>This is speculating about what the brain is doing. You are
>making a rather simple procedure (comparing stuff) into
>rocket science. If there are audible differences, those
>differences should show up in a comparison procedure that
>involves just two things: the ears and the brain. Note that
>I am not leaving the brain out as you seem to believe
>happens with a blind comparison. Rather, I am eliminating a
>variable - knowledge of what is playing - that will possibly
>confuse the brain when it comes to what the ears are sending
>to it in the way of rather straightforward information.
>
Nice try, Howard, but it doesn't work that way. You are taking a complex
procedure (how the brain processes information) and are trying to simplify it
to justify your belief in blind testing.
Regards,
Mike
Arny Krueger
>> Howard Ferstler fer...@attglobal.net wrote:>
>> Perhaps you can. However, there is only one way to validate
>> your claims for sure: do the comparisons blind.
> So you say. If the blind test interferes with the results, then
> it's not valid. Can you understand that?
Whether a blind test interferes with the results AFAIK has never been
scientifically proven. It's just idle speculation from people who as
a rule, never do blind tests.
OTOH, it is well known that sighted evaluations involving small
differences interferes with the results of the test. This is a
scientific fact, one that has been proven many times. Why is it that
the interference that a sighted evaluations cause is not immediately
used as a reason to not do them?
Mkuller
wrote:>
>This sure was a long post, and this certainly was a long thread, for
>something so irrelevant.
>
If that's so, why did you post so many times?
>Why all the interest in blind testing? Has someone been pushing it for
>audiophiles?
Stewart Pinkerton claims it's the only way to know what you are
really hearing and then there are Arny Krueger's abx infomercials for
a start.
>Why does this topic keep coming up so often here?
It takes two sides for an arguement to occur...
Regards,
Mike
Emerson Wood
> emers...@yahoo.com (Emerson Wood) wrote in message news:<ak31a...@enews2.newsguy.com>...
> > Howard Ferstler <fer...@attglobal.net> wrote in message news:<ajr4h...@enews2.newsguy.com>...
>
> > > How would you know this is true? You know what is playing,
> > > and so you could easily be fooled by your own
> > > preconceptions. It is fatuous to claim that because
> > > differences disappear when listening blind those differences
> > > must somehow be obscured by the blind-comparison protocol.
> > >
> > > Most people would say just the opposite: sighted comparisons
> > > introduce potential prejudices that blind comparisons will
> > > not allow to be introduced.
>
> > Right--blind comparisons just have their own prejudices.
>
> And what might those be? The fact is that no matter what you might
> think about blind comparisons screwing up one's perceptive
> capabilities, at least the results cannot be fabricated. You either
> can or cannot hear differences, and there is no way to pick out your
> favorite amp or set of wires because they happen to be your favorite.
> You have to judge by sound alone. You cannot be prejudiced one way or
> the other, because you have no idea which component is playing during
> the blind phase of the test.
In answer to your question, I listed some of the mechanisms
in my post which you snipped. The prejudices would be to bias
the result either (a) toward "no difference" or (b) toward hearing
a marked difference that is relatively irrelevant to *music*
listening.
>
> This is in contrast to a sighted comparison of any kind, where there
> is no way to know if the results of the test were because genuine
> differences were audible or if the knowledge of what was playing
> somehow colored the perceptions of the participant and caused him to
> pick the product that he subconsciously preferred.
> And there is no way the participant can prove to anyone else, or in my
> opinion even to himself for absolutely sure, that he was actually
> hearing differences during a sighted comparison that would not show up
> audible differences during a DBT with the same components.
As I stated in my post which you clipped, you keep saying over
and over that in sighted tests "there is no way to know,"
"no way to prove", no way to be "abolutely sure," as if that were
a problem, as if knowledge were only useful to the extent that it is
certain.
You are also very set on this "sighted vs. blind" dichotomy, and as
I've hinted here in the past, life is much richer than "sighted
vs. blind." In reality, with sensitivity to the mechanisms of
aesthetic perception, we can do sighted tests that are less biased,
and blind tests that are more sensitive. It is always "DBT as
commonly done" that I identify as the problem.
>
> Sighted comparisons always result in the participant not ever being
> absolutely sure of what he is hearing. He may feel good about the
> results for a while, but down deep inside there will always be that
> nagging doubt. Well, I suppose that is why upgrading is so common in
> the high-end community.
>
> Howard Ferstler
There's always a nagging doubt. Even with controlled scientific
experiments, you have to wonder whether you controlled everything
and whether your results are useful in making real-life decisions.
My armchair analysis of you is that you can't abide "doubt," so
you prefer methods that give the illusion of certainty.
-Emerson
Mkuller
>>Following your (biased) "logic", how do you know that you are really
>>"hearing" those differences, gross or not, if you're listening is
>>sighted? Why wouldn't you apply the same standards to all of your
>>listening?
>
>Stewart Pinkerton responded:>
>Simple. If you have any doubts, you do a blind test. When you've
>scored 100% so many times that it's become boring, you don't waste
>your time any more.
You may hear a lot of different things comparing speakers, but
according to your logic, they may all be figments of your imagination
unless you show you can hear every single difference blind. The fact
that there were gross differences enough that you could chosse them
correctly doesn't mean everything your heard sighted is valid. Or
does it?
Regards,
Mike
S888Wheel
>>>bigger deal with a blind test than with a sighted test, but obviously
>>>those who are sweating the status of their golden ears or the
>>>performance of a favorite set of wires or amp will probably have a
>>>rather unsettled state of mind during the test. Life is tough.
>>
>
>>This reminds me of people who wonder why people with Terets simply
>>just don't do the things they do. The human mind is a complex system.
>>It is a well known fact that seemingly related tasks such as writing
>
>>language and speaking it use very different parts of the brain. You
>>can't just will it away.
>
>
>This is precisely the problem with open evaluation for sound quality
>differences. There are human response bias mechanisms (level/quality
>confusion, pro-choice decision tendency are two) that cannot be
>willed away.
The question at hand is whether or not this is a problem with ABX
DBTs. Critiques of sighted tests, while certainly valid, don't
really address the question at hand.
>>If the task of doing ABX DBTs is masking
>>something in our perceptions it is a problem. That this is a problem
>>with ABX listening tests hasn't been proven but has it been
>>disproven?
>
>
>There is no evidence that controls for human listening response bias
>affect anything except non-sonic bias mechanisms.
>
Is there any evidence that it doesn't?
>>So any assumptions either way are just that, assumptions.
>>Unless there have been some tests I don't know about that addressed
>>this issue. It would be very easy to test whether or not the brain
>>changes gears in ABX DBTs although I suspect it would be expensive.
>
>Even if brain activity were found to change that is not evidence that
>any sonic factors would have been masked.
I agree but what this test could offer is evidence in regards to the
propostion that ABX DBTs changes the "state of mind" of the testee.
If such tests showed a significant change in brain activity then the
propostion is not so outrageous, if such tests showed little or no
measurable changes in brain activity then the proposition looses
credibilty.
The answer I think is obvious. Do controled blind tests of amplifier
and/or wire comparisons that allow "subjectivists" to do sighted
listening tests in the same manner as they do when they claim to hear
differences that "objectivists" feel are unreliable do to the lack of
propper controls.
One way would be to have subjectivists audition two amps (believed by
subjectivists to sound very differen)t using sighted tests. The amps
would be "hidden" in identical looking chasis that are only
differentiated by a visual designation that is unlikely to create a
bias. Have the "subjectivist" audition the amps as he or she normally
would and choose a preference. Then repeat this with two new amps
that are in actuality the same two amps with new designations. Repeat
this test until you have an adequate sample. Even though this might
be seen as "ambush" testing there is certainly nothing I can see that
would interfere with the "subjectivist" conducting listening tests
in pretty much the same way as they prefer to. And you can do this as
a controled DBT.
ludovic mirabel
See Mike's message below:
Dear Mike, you're trying to describe what individual *minds* make of
musical experience to an audience hung up on the point -of-entry: the
ears. You see, they feel something can be "measured" at that level.
That makes them feel at home and safe.
Might as well talk about poetry to illiterates. They'll ask for the
length and number of the lines that make "poetry".
Bias exists. Those raised on the marketing hype can think only of one
form of it: prejudice for the bigger. shinier and more expensive.
Blinding helps with that.
But there is much more. We all have built in, individual bias- our
mind- a unique mixture of inborn and acquired. That is where music is
processed.
You can measure "thresholds of audibility" for frequencies, distortion
, volume etc, You can not measure how our individual minds will handle
differences in music reproduction by musical instruments, whether
electraonic or "acoustic". It is obvious to you and me but you'll
never convey it to the "I can measure it or it doesn't exist" crowd",
They'll want a "test". And not just any test. It has to be a switched
one: A then B then X to compare with A and B.
Next you get an "expert panel" composed of people with different
hearing ability. musical experience, ability and training. different
likes and dislikes and expose them to your switched test- a test that
has been abundantly shown by its accredited sponsors to divide
listeners into the ABX-able, less able and unable, period. Obeying
your objectivist bias you now ignore the *only interesting results*-
those of the outstanding performers- shake and stir and get another
null=" They all sound the same" result.
You proved to your satisfaction that what you can't hear doesn't
exist.
I've been told that I'm going on too long. Its true but that is
because like you I can't bring myself to believe that ignoring all
evidence people will dress up into a pointed hat, climb on a chair and
shout "I'm a scientist and you're a dirty rascal. Now disprove it"
Ludovic Mirabel
Nousaine
This point is absolutely correct. Which is why I use additional control
techniques such as screens where feasible, cd-r collections of programs which
helps routinize presentation and helps ensure use of high quality and identical
programs. And especially some type of reasonable level normalization.
Nousaine
...snip.....
>
>>
>>>language and speaking it use very different parts of the brain. You
>>>can't just will it away.
>>
>>This is precisely the problem with open evaluation for sound quality
>>differences. There are human response bias mechanisms (level/quality
>>confusion, pro-choice decision tendency are two) that cannot be
>>willed away.
>
>The question at hand is whether or not this is a problem with ABX
>DBTs. Critiques of sighted tests, while certainly valid, don't
>really address the question at hand.
They precisely address those two problems with level matching and blind
protocols. That's why they are used.
>>>If the task of doing ABX DBTs is masking
>>>something in our perceptions it is a problem. That this is a problem
>>>with ABX listening tests hasn't been proven but has it been
>>>disproven?
So far people have shown that ABX testing is sensitive to any sound
reproduction errors that have been shown and are known to be audible.
The argument boils down to arguments that controlled listening tests haven't
been shown to be sensitive to reproduction differences that come from causes
that have so far NOT been shown to have audible consequence.
So I see this coming down to ABX testing would have to show audibility in an
experiment where there is no currently recognized audible cause.
It's just a mirror of the "You Can Never Prove the Null Hypothesis" argument.
>>
>>There is no evidence that controls for human listening response bias
>>affect anything except non-sonic bias mechanisms.
>>
>
>Is there any evidence that it doesn't?
How would one show that? What happens is that people often identify audible
consequence of nominally equivalent devices under open conditions which just
disappear when nothing more than forcing a choice when the answer is not known
in advance....even under ABX where a subject has sight of both alternatives all
the time.
A few years ago I tested the idea that devices that cannot be differentiated by
themselves may have audible consequence when used in series. So I constructed
two systems one of which was decidedly what i called Tweak using an outboard
DAC, a vacumn tube preamplifier, a high-end Bryston power amplifier, expensive
networked Monster speaker cables, $100 a meter interconnects, spikes everywhere
they would work, Teknisonic vibration dampers to isolate the speaker cables
from the floor and very careful cable dress.
The other was definitely Geak with a 20 year old solid state preamplifier built
from a low priced kit, a used $200 power amplifier, interconnects randomly
drawn from a junk box, 16-gauge car speaker cable (zip cord) with a 6-foot
length for one channel and a 25-foot length for the other with the cable
wrapped around the power cords several times.
The digital outputs of a Marantz Cd-player drove the Tweak system and the
analog ouputs of the same cd-player drove the Geak system.
Once installed I found the systems both had response that was within 0.2 dB of
each other but the Tweak system had a 2-dB channel imbalance which I
compensated with the balance control on the Geak system.
Using an ABX comparator I was unable to hear any difference between them
myself. I then invited a number of enthusiasts one at a time to audition them
A/B with programs that they had personally found to be the most revealing for
finding audibility differences.
ALL of them said they thought they sounded different. But NONE of them were
able to differentiate when nothing more than a blanket was covering the inputs
on the rear of the PSB Stratus Mini speakers (reference device with
measurements taken in the NRC) hiding which set of speaker wires were connected
to the speakers.
Someone following this debate might then conclude that bias-controls obscured
differences that were audible under sighted conditions. But actually all the
sighted information, except the answer to the question, was always available.
The ONLY factor introduced was the elimination of a bias mechanism, subjects
are inclined to report differences that do not have audible source, that is
known to have consequence.
In this experiment I tried to include every item that I had heard people report
as having audible consequence, not directly tied to frequency response or other
known audible atrifact, either by itself or in tandem.
>
>>>So any assumptions either way are just that, assumptions.
You are the only person making assumptions on this point. IMO it doesn't matter
of bais controls affect brain activity because there is no evidence that I know
fo that would tell us that brain activity is related to acoustical hearing
sensitivity.
>>>Unless there have been some tests I don't know about that addressed
>>>this issue. It would be very easy to test whether or not the brain
>>>changes gears in ABX DBTs although I suspect it would be expensive.
>
>>
>>Even if brain activity were found to change that is not evidence that
>>any sonic factors would have been masked.
>
>I agree but what this test could offer is evidence in regards to the
>propostion that ABX DBTs changes the "state of mind" of the testee.
Who cares? What I care about is whether sensitivity to audible artifacts is
maximized and control over extraneousl variables not related to audibility are
controlled.
>If such tests showed a significant change in brain activity then the
>propostion is not so outrageous, if such tests showed little or no
>measurable changes in brain activity then the proposition looses
>credibilty.
The proposition is not outrageous at all. I would guess that any decision
either sighted or blind affects brain activity. Why shouldn't it? Who cares,
unless it actually masks audibility ... and there is no evidence that this is
the case.
>The answer I think is obvious. Do controled blind tests of amplifier
>and/or wire comparisons that allow "subjectivists" to do sighted
>listening tests in the same manner as they do when they claim to hear
>differences that "objectivists" feel are unreliable do to the lack of
>propper controls.
Yes.
>One way would be to have subjectivists audition two amps (believed by
>subjectivists to sound very differen)t using sighted tests. The amps
>would be "hidden" in identical looking chasis that are only
>differentiated by a visual designation that is unlikely to create a
>bias. Have the "subjectivist" audition the amps as he or she normally
>would and choose a preference. Then repeat this with two new amps
>that are in actuality the same two amps with new designations. Repeat
>this test until you have an adequate sample. Even though this might
>be seen as "ambush" testing there is certainly nothing I can see that
>would interfere with the "subjectivist" conducting listening tests
>in pretty much the same way as they prefer to. And you can do this as
>a controled DBT.
Actually David Clark's Black Box test was an example of this.
Cannot Understand Why You Dis AOL
From: mku...@aol.com (Mkuller)
>Date: 08/25/2002 12:38 PM Eastern Daylight Time
>Message-id: <akb16...@enews3.newsguy.com>
>
>> Gary Eickmeier geic...@tampabay.rr.com
>wrote:>
>>This sure was a long post, and this certainly was a long thread, for
>>something so irrelevant.
>>
>
>If that's so, why did you post so many times?
>
>>Why all the interest in blind testing? Has someone been pushing it for
>>audiophiles?
>
>Stewart Pinkerton claims it's the only way to know what you are
>really hearing
er..he's not alone , you know.
He has science on his side too
The question is, how much can we reliably know of audible difference, from
sighted comparison?
I'm comfortable with the fact that my sighted evaluations of sound quality are
prone to bias if I haven't done the proper controls. I readily admit it.
It doesn't keep me from purchasing components *without* doing blind
comparisons. Because 1) if it sounds good *to me*, that's good enough *for me*
and 2) sound is not always the only consideration for my purchases.
There is a difference between deciding that a component sounds good to you, and
making objective claims about its sound re: another component. When I do the
latter I try to note whether I have validated my claim in any fashion or not.
And even if I sometimes fail to be rigorous in my writing about such claims, I
am always ready to admit that a blind comparison might show that my sighted
perception was illusory.
The 'interest' in blind testing comes from the many, many, many claims of
audible difference that populate audiophilia, and more importantly are
presented as *fact*, without proper validation.
_______
-S.
Stewart Pinkerton
No, but it means that an ABX test is pointless in this case, since you
will score 100%.
Howard Ferstler
> > > Right--blind comparisons just have their own prejudices.
> > And what might those be? The fact is that no matter what you might
> > think about blind comparisons screwing up one's perceptive
> > capabilities, at least the results cannot be fabricated. You either
> > can or cannot hear differences, and there is no way to pick out your
> > favorite amp or set of wires because they happen to be your favorite.
> > You have to judge by sound alone. You cannot be prejudiced one way or
> > the other, because you have no idea which component is playing during
> > the blind phase of the test.
> In answer to your question, I listed some of the mechanisms
> in my post which you snipped. The prejudices would be to bias
> the result either (a) toward "no difference"
The only way this would make sense would be if the
participant did not care about high-fidelity sound
reproduction. To say that he heard no differences when he
did hear differences would mean that he wanted to force the
industry towards making substandard products. While some
individuals might want to do that (although I do not know
who they might be or why they would want that), it is
unlikely that anyone with a real interest in hi-fi would
want that to happen. It is wrong to say that those who claim
to not hear differences want to drag down the quality of
hi-fi sound gear.
> or (b) toward hearing
> a marked difference that is relatively irrelevant to *music*
> listening.
Well, I am talking about not hearing differences, rather
than hearing differences. Obviously, if someone hears no
differences that is just about "it" for them, no matter if
they are listening to music, test signals, random-noise
signals, or just plain noise. The bottom line is: can they
hear differences? If they cannot, then I cannot see what
else is left to say. For that individual, there are no
differences.
On the other hand, those who claim to hear differences seem,
in my opinion, obligated to make sure of that fact,
particularly if they are trying to prove something to others
that involves those others spending money on products. Of
course, maybe the others do not care. In that case, they
obviously can spend as they please.
> > This is in contrast to a sighted comparison of any kind, where there
> > is no way to know if the results of the test were because genuine
> > differences were audible or if the knowledge of what was playing
> > somehow colored the perceptions of the participant and caused him to
> > pick the product that he subconsciously preferred.
> > And there is no way the participant can prove to anyone else, or in my
> > opinion even to himself for absolutely sure, that he was actually
> > hearing differences during a sighted comparison that would not show up
> > audible differences during a DBT with the same components.
> As I stated in my post which you clipped, you keep saying over
> and over that in sighted tests "there is no way to know,"
> "no way to prove", no way to be "abolutely sure," as if that were
> a problem, as if knowledge were only useful to the extent that it is
> certain.
Well, I would like to think that a concerned audio
enthusiast might want to at least be reasonably certain. It
looks to me as if any sighted comparison will always leave
open a good number of uncertainties. The individual may like
a certain amp or certain set of wires, but if he likes them
because of what he heard during a sighted comparison that
will always leave open the possibility that he was
overlaying his preconceptions. That may more than satisfy
him, and if that is the case more power to him. After all,
it is his happiness that matters.
However, I get the impression that one reason so many high
enders upgrade the way they do is that they are never sure.
They continually hope to improve things, and one reason they
are continually hoping is that they never try really hard to
determine if something really sounds better, or even
different. Clearly, certain manufacturers who depend upon
people upgrading to their products have a vested interest in
sloppy comparisons.
> You are also very set on this "sighted vs. blind" dichotomy, and as
> I've hinted here in the past, life is much richer than "sighted
> vs. blind."
Obviously, it is. However, we are not really doing something
all that complex. All we are doing is comparing for
differences. It is not rocket science, or theology, or
poetry, or any one of any number of complex and
multi-dimensional tasks. It is just comparing gear for sonic
differences if the differences are small or comparing for
preference if they are large enough to be clearly audible
and suitable for a preferential evaluation.
> In reality, with sensitivity to the mechanisms of
> aesthetic perception, we can do sighted tests that are less biased,
> and blind tests that are more sensitive. It is always "DBT as
> commonly done" that I identify as the problem.
Well, the most commonly done DBT is the ABX test, I assume.
Certainly, it is the one that is most written about. I think
it is a good procedure, since is combines a sighted protocol
(comparing A to B) with an blind protocol (compare A or B to
"X"), and allows the participant to do so at his own rate of
speed, with his own software, and hopefully with is own
ancillary hardware. That may not always be possible, of
course, but if it is done I cannot see any way that is
fairer or more conclusive.
Frankly, I cannot see how a sighted test can be anything but
questionable, particularly when comparing items that
supposedly have great differences that disappear when the
comparisons are don blind. This is not preference we are
talking about, where there are obvious differences.
> > Sighted comparisons always result in the participant not ever being
> > absolutely sure of what he is hearing. He may feel good about the
> > results for a while, but down deep inside there will always be that
> > nagging doubt. Well, I suppose that is why upgrading is so common in
> > the high-end community.
> There's always a nagging doubt. Even with controlled scientific
> experiments, you have to wonder whether you controlled everything
> and whether your results are useful in making real-life decisions.
> My armchair analysis of you is that you can't abide "doubt," so
> you prefer methods that give the illusion of certainty.
In some areas I prefer as much certainty as possible. I can
get that with wires and amps if I do a DBT. That is
conclusive, as far as I am concerned.
When comparing speakers, I have all sorts of caveats,
because one minute one sounds better and the next minute the
other may sound better. They are different, but different in
a multitude of easily heard ways that mysteriously change as
the source material changes.
I do not find this with decently made amps and wires. They
always sound the same to me - excepting if an underpowered
amp is driven into overload.
Howard Ferstler
Howard Ferstler
>
> mku...@aol.com (Mkuller) wrote in message news:<ak90r0$64i$1...@bourbaki.localdomain>...
> See Mike's message below:
>
> Dear Mike, you're trying to describe what individual *minds* make of
> musical experience to an audience hung up on the point -of-entry: the
> ears. You see, they feel something can be "measured" at that level.
> That makes them feel at home and safe.
They are not measuring anything. They are just doing careful
comparing. On the other hand, you appear to be setting
yourself up as a reference standard for what certain
products sound like. What proof can you offer that your
reference-standard status is truly a reference standard? And
how can it objectively be proven that what you hear is
really happening? All we have is your word for it.
At least a DBT offers researchers a way to do repeatable
experiments. Your personal experiences work only for you,
and nobody is going to be able to validate your claims. Of
course, for you all that matters are those personal
experiences. However, there is no way to export those
experiences. At least with a DBT we can get some idea of
what is and what is not important in amplifier and wire
performance.
> Might as well talk about poetry to illiterates. They'll ask for the
> length and number of the lines that make "poetry".
Again, the DBT is not poetry, rocket science, physics, or
art. It is a simple comparison procedure. Not complex, not
arcane, and not misleading. You either hear differences or
you do not. If you do not, you either live with that fact or
make up excuses.
> Bias exists.
Thereby justifying the use of the DBT to keep those biases
from screwing up the comparisons.
> Those raised on the marketing hype can think only of one
> form of it: prejudice for the bigger. shinier and more expensive.
> Blinding helps with that.
For sure. That is one of the rationales for using the
procedure.
> But there is much more. We all have built in, individual bias- our
> mind- a unique mixture of inborn and acquired. That is where music is
> processed.
But you are not processing music with a DBT. You are just
comparing the performance of things that supposedly should
perform the same. They either sound the same or they do not.
This is not a big deal at all. It is a simple procedure that
is only being made complex by you.
> You can measure "thresholds of audibility" for frequencies, distortion
> , volume etc, You can not measure how our individual minds will handle
> differences in music reproduction by musical instruments, whether
> electraonic or "acoustic".
The DBT is not measuring anything. It just allows you to
determine if differences are audible. Not complex, not
arcane, and not even tedious, once the hardware is set up.
Just listen and make your choices.
> It is obvious to you and me but you'll
> never convey it to the "I can measure it or it doesn't exist" crowd",
> They'll want a "test". And not just any test. It has to be a switched
> one: A then B then X to compare with A and B.
Right. And the results can be repeated in other tests, done
by other individuals. The idea is not to determine
preference. The idea is just to compare for differences. If
they are there, you will hear them. If they are not, you
will not hear them. If you do not hear them, you will either
have learned something or will make excuses about why a
blind comparison is no substitute for mysterious,
unfathomable knowledge of which products are best.
> Next you get an "expert panel" composed of people with different
> hearing ability. musical experience, ability and training. different
> likes and dislikes and expose them to your switched test- a test that
> has been abundantly shown by its accredited sponsors to divide
> listeners into the ABX-able, less able and unable, period.
You miss the point. With such a test, if an individual
cannot hear differences, then for him those differences do
not exist. It does not matter how expert they are or how
good their hearing is. If they cannot hear differences, that
is it and that is all that matters.
> Obeying
> your objectivist bias you now ignore the *only interesting results*-
> those of the outstanding performers- shake and stir and get another
> null=" They all sound the same" result.
If they all sound the same to an individual, that is all
that matters. Well, what also matters is that an individual
learn to live with that information.
> You proved to your satisfaction that what you can't hear doesn't
> exist.
Right. All that matters is what the individual hears. If he
is not trained that does not matter. What matters is whether
amps or wires or CD players sound the same or different to
him. If they sound the same, he is in a position to save
himself some money and shopping headaches.
> I've been told that I'm going on too long. Its true but that is
> because like you I can't bring myself to believe that ignoring all
> evidence people will dress up into a pointed hat, climb on a chair and
> shout "I'm a scientist and you're a dirty rascal. Now disprove it"
Nevertheless, you have yet to prove that setting yourself up
as a reference standard for sound quality is superior to
doing a DBT. It may make you happy, and that is fine for
you. However, your experiences certainly cannot be exported
to other people. And if you want to prove to people that you
can hear differences between certain products, the only
surefire way to do it is with the DBT.
While a DBT done by other people is no surefire way for an
individual to know if the procedure is fully meaningful for
him, it certainly gets him further into the ballpark than
the technique you offer.
Howard Ferstler
ludovic mirabel
Dear Mr, Krueger, I wish you weren't so touchy.
No one denies that you're perfotming an admirable service in helping
some to remove one kind of bias- the crudest and most obvious to the
generation raised on marketing hype. PROVIDING they are ABX able
and/or trained, I happen to feel that for me , and some others, ABX
without training holds little interest and I suspect that as in my own
case the training effort would outweigh any possible benefit because
I'm not an ABX natural. I have another method to achieve marketing
bias removal which suits me better but might not suit others. You call
it "competing", I call it to each his own.
Mike says in essence that testing and guessing demands a special kind
of attention which is unlike normal, prolonged listening for detecting
quality features. Again this varies from individal to individual.
Documentation apart from being common sense is plentiful. Why else
would you offer training?
I like some testing alternatives and best of all I like *listening*.
Listening to what a given component does to the sound of a soprano
voice. violins , piano, cello, drum etc. I do some tweaking and I
recognise I'm biased in favour of the latest tweak I made, I get then
a single blind (and no spoken comments) opinion of a ruthless critic I
have at home with younger and better ears thanmine, who likes music in
the live, plays the piano competently and doesn't care for hi-fi
fiddling one little bit,
Only reluctantly and very occasionally I use my left-right method. I
do not tell your satisfied customers to change- in fact I don't care
if anyone that I don't know does, Listening to music is a private
pleasure just like reading a favourite novel, I want to share it with
my friends and I don't care about the rest of the world. You
apparently do and since you seem to derive no material benefits from
it all you must be doing it out of love of humanity.
Who would carp at that?
Ludovic Mirabel_______________________________________________
Howard Ferstler
> Mike says in essence that testing and guessing demands a special kind
> of attention which is unlike normal, prolonged listening for detecting
> quality features. Again this varies from individal to individual.
> Documentation apart from being common sense is plentiful. Why else
> would you offer training?
Training is irrelevant. All that matters is whether the individual,
trained or untrained, can or cannot hear differences. Satisfying the
individual enthusiast is all that matters in the long run.
Unfortunately for your procedure, "prolonged listening for detecting
quality features" offers no assurances (either to those wanting
information about what you have heard or for you, either) that
preconception bias will not intrude.
Howard Ferstler
Cannot Understand Why You Dis AOL
>Message-id: <akb17...@enews3.newsguy.com>
>
>>>mkuller wrote:>>
>>>Following your (biased) "logic", how do you know that you are really
>>>"hearing" those differences, gross or not, if you're listening is
>>>sighted? Why wouldn't you apply the same standards to all of your
>>>listening?
>>
>>Stewart Pinkerton responded:>
>>Simple. If you have any doubts, you do a blind test. When you've
>>scored 100% so many times that it's become boring, you don't waste
>>your time any more.
>
>You may hear a lot of different things comparing speakers, but
>according to your logic, they may all be figments of your imagination
>unless you show you can hear every single difference blind
Yup.
If you wanted to report your result with the most experimental rigor
and fewest assumptions, you'd use a blind test protocol.
(However, scientists are not generally required to reinvent the wheel
with each experiment.)
_______
-S.
Nousaine
Actually the normal human bias is to report different even when given
identical alternatives. You can see this is the Stereophile tests
where subjects would answer 'different' about twice as often as
'same.' . Also check "Can You Trust Your Ears?" AES Preprint and
Stereo Review where it can be seen that subjects will report
different over 3/4 of the time when given two identical sound
presentations.
This is one of the prejudices that actually exists and can be shown
to exist.
[quoted text deleted -- deb]
Bob Marcus
> >Howard Ferstler fer...@attglobal.net
> wrote:>
> >Perhaps you can. However, there is only one way to validate
> >your claims for sure: do the comparisons blind.
> >
> So you say. If the blind test interferes with the results, then it's not
> valid. Can you understand that?
In other words, if you do a comparison two ways, and get different
results, Mike decides which is right.
> >There is no fathoming the subconscious mind.
>
> Sometimes the "conscious" mind is difficult to explain as well.
>
> >That is why
> >blind comparisons are important if hair-splitting details
> >are important to you or you are trying to prove claims to
> >outsiders or do genuine research on the topic.
> >
>
> What you call "hair-splitting detail" are what High End audio is all about.
> With mid-fi I agree they aren't important. Remind me which it is you deal
> with.
Careful. Argument by snobbery doesn't become you.
> Obviously, you have accepted blind testing without question. Blind testing
> will work fine when the differences are large. For subtle differences, blind
> testing causes the differences to be obscured.
Ah, argument by assertion. Much better.
> >> >You have
> >> >no way to prove to anyone, including yourself, if the
> >> >differences you think you heard were really audible.
>
> >> So it's proof you're after. How can you prove those subtle differences you
> >> hear in sighted listening that cannot be identified or recalled from your
> >> memory during the blind "decision-making process" aren't really there, and
> not
> >> obscured by how your brain processes information?
> >
> You are speculating that sight interferes with what you are hearing.
Who said that? It may embellish what you are hearing, but that's a
different matter altogether.
> It can
> but it doesn't always. Blind testing appears to always obscure subtle audible
> differences because of the "decision-making" process involved.
And what evidence can you provide that it is specifically because of
the "decision-making process involved"? None that I've seen. Argument
by assertion again.
>Some blind
> listeners describe trying to identify X as A or B as confusion or just stress.
So many listeners, so many excuses. So let's find a few who aren't
bothered by these problems, and see if they can hear the differences
you claim exist. There must be a few. Where are they?
> >This is speculating about what the brain is doing. You are
> >making a rather simple procedure (comparing stuff) into
> >rocket science. If there are audible differences, those
> >differences should show up in a comparison procedure that
> >involves just two things: the ears and the brain. Note that
> >I am not leaving the brain out as you seem to believe
> >happens with a blind comparison. Rather, I am eliminating a
> >variable - knowledge of what is playing - that will possibly
> >confuse the brain when it comes to what the ears are sending
> >to it in the way of rather straightforward information.
> >
> Nice try, Howard, but it doesn't work that way. You are taking a complex
> procedure (how the brain processes information) and are trying to simplify it
> to justify your belief in blind testing.
I'd say you're taking a complex procedure you know precious little
about and inventing baseless hypotheses about it, because you don't
like the implications of the more grounded hypotheses offered by
people who study said procedure for a living. What you're doing is
rationalizing. Very human.
bob
Arny Krueger
news:akcev5$nia$1...@bourbaki.localdomain
> I happen to feel that for me,
> and some others, ABX without training holds little interest and
> I suspect that as in my own case the training effort would
> outweigh any possible benefit because I'm not an ABX natural. I
> have another method to achieve marketing bias removal which suits
> me better but might not suit others. You call it "competing", I
> call it to each his own.
This is an ironic paragraph. It contains numerous negative statements
about a methodology for subjective testing that the author has
publicly stated that he wants to provide a competitor for, but ends
with the phrase "To each his own". Obviously, if this matter was
really being left to each person to decide for himself, the author
would not be making negative statements about a competitive
methodology!
> Mike says in essence that testing and guessing demands a special
> kind of attention which is unlike normal, prolonged listening for
> detecting quality features.
I can agree with half of this. Testing demands a different kind of
attention that is unlike normal listening for enjoyment. OTOH, hairs
are obviously being split in this paragraph because "prolonged
listening for detecting quality features" is just another kind of
testing that is also dissimilar for listening for enjoyment.
>Again this varies from individual to
> individual. Documentation apart from being common sense is
> plentiful. Why else would you offer training?
I offer training because listening is a skill that can be developed
for greater sensitivity and reliability, whether sighted or blind.
Blind testing provides a good working environment for listener
training because it provides a clear-cut indication of the listener's
progress.
> I like some testing alternatives and best of all I like
> *listening*.
I like to listen for enjoyment, too.
I also like to listen analytically to better appreciate the artistry
and skill of the composers and musicians. This is not the same as
listening for enjoyment, and it is something that is based on formal
training as well.
Howard Ferstler
> >Perhaps you can. However, there is only one way to validate
> >your claims for sure: do the comparisons blind.
> So you say. If the blind test interferes with the results, then it's not
> valid. Can you understand that?
We have both inadvertently cut to the chase. As I stated in
a previous post, you appear to have set yourself up as a
reference standard, and have done so by using sighted
comparisons.
When a procedure (the DBT) comes along and invalidates your
conclusions about the sound of certain products, rather than
re-evaluate your own procedures and see if the new procedure
offers some new insights into what is going on, you dismiss
the new procedure and continue to hold yourself up as a
reference standard.
I cannot see any other way of interpreting virtually
everything you have written about double-blind A/B testing
in this series of threads and in a number of others as well.
What can you offer us as proof of your contentions about
double-blind comparing other than your feelings that what
you hear is a reference standard?
Howard Ferstler
Robert Lang
> >
> differences interferes with the results of the test. This is a
> scientific fact, one that has been proven many times. Why is it that
> the interference that a sighted evaluations cause is not immediately
> used as a reason to not do them?
The concept of Blind A/B testing and perhaps the track record of Blind
A/B testing is impressive. But how practical/realistic is it for the
vast majority of consummers/audiophiles to employ Blind A/B testing
for specific components, even if they were inclined to use the
process? Does it not involve training to properly utilize the
procedure? Or is Blind A/B testing most effectively used by
manufacturers and researchers for development of their own products,
or perhaps by a few reviewers. What reviewers regularly utilize and
publish Blind A/B testing results. And how do we know that their
testing is valid because Blind A/B testing is not so simple, or so it
seems (it seems like there are a lot of way to do Blind A/B testing
incorrectly)? How can the consummer actually utilize this information
without actually ingaging in the process? How can the cross section of
participants of this board, for example, effectively use Blind A/B
testing? A couple of years ago I wanted to compare two amps I was
considering, the Musical Fidelity Nu Vista 300 and a Spectural model,
how could I have practically utilized Blind A/B to arrive at a
decision? Is Blind A/B testing meant for participation by the cross
section of audiophiles.
Sorry for all the questions. That truly was not my intent. The
questions kept coming as I was writing.
Robert C. Lang
S888Wheel
>>DBTs. Critiques of sighted tests, while certainly valid, don't
>>really address the question at hand.
>
>
>They precisely address those two problems with level matching and blind
>protocols. That's why they are used.
>
I'm afraid things have been taken out of context. Level matching and
blind protocals do nothing to insure that ABX tasking does not lead
to masking due to differences in brain activity.
>>>>If the task of doing ABX DBTs is masking
>>>>something in our perceptions it is a problem. That this is a problem
>>>>with ABX listening tests hasn't been proven but has it been
>>>>disproven?
>
>
>So far people have shown that ABX testing is sensitive to any sound
>reproduction errors that have been shown and are known to be audible.
>
How? What has been done to determine everything that is "known to be
audible"? I certainly hope the answer isn't DBTs. If it is you have a
problem.
>The argument boils down to arguments that controlled listening tests haven't
>been shown to be sensitive to reproduction differences that come from causes
>that have so far NOT been shown to have audible consequence.
>
Well what constitutes a controled listening test is in and of itself
another issue.
I believe the issue isn't controled listening tests but specifically
DBTs done in the fashion of the ABX test. Again verification that
DBTs are omnisensitive to all human audible perception by use of
other DBTs is fatally flawed.
>So I see this coming down to ABX testing would have to show audibility in an
>experiment where there is no currently recognized audible cause.
I disagree. I think it comes down to doing tests that do not
interfere with the methods that subjectivists feel work in discerning
audible differences that are claimed to be masked in ABX type DBTs.
>>Is there any evidence that it doesn't?
>
>
>How would one show that? What happens is that people often identify audible
>consequence of nominally equivalent devices under open conditions which just
>disappear when nothing more than forcing a choice when the answer is not
>known
>
>in advance....even under ABX where a subject has sight of both alternatives
>all
>the time.
I thought my test proposal was a decent idea on how to show it.
I think if one wanted to be really thourough the best thing would be
to do the test I suggested with amplifiers in conjuction with ABX
DBTs, sighted tests, brain activity measurements during sighted and
ABX DBTs and null tests on the amplifiers themselves. I think the
data from such a combination of tests would say a lot about this
issue
>>>>So any assumptions either way are just that, assumptions.
>
>
>You are the only person making assumptions on this point. IMO it doesn't
>matter
>of bais controls affect brain activity because there is no evidence that I
>know
>fo that would tell us that brain activity is related to acoustical hearing
>sensitivity.
Quite the opposite. I have merly suggested that the claims of
subjectivists that ABX DBTs mask real audible differences be
investigated. I have made no assumptions about what the results would
be
>>>>Unless there have been some tests I don't know about that addressed
>>>>this issue. It would be very easy to test whether or not the brain
>>>>changes gears in ABX DBTs although I suspect it would be expensive.
>
>>>Even if brain activity were found to change that is not evidence that
>>>any sonic factors would have been masked.
>>
>
>>I agree but what this test could offer is evidence in regards to the
>>propostion that ABX DBTs changes the "state of mind" of the testee.
>
>Who cares? What I care about is whether sensitivity to audible artifacts is
>maximized and control over extraneousl variables not related to audibility
>are
>controlled.
Who cares? Anybody who suspects that ABX DBTs are masking ones
ability to differenciate audible differences that are heard under
conditions that subjectivists feel don't mask such differences might
care. Obviously you don't think it matters but that is an assumption.
No matter how reasonable your assumtion seems to be (It seems
reasonable to me) it isn't verified.
>>If such tests showed a significant change in brain activity then the
>>propostion is not so outrageous, if such tests showed little or no
>>measurable changes in brain activity then the proposition looses
>>credibilty.
>
>
>The proposition is not outrageous at all. I would guess that any decision
>either sighted or blind affects brain activity. Why shouldn't it? Who cares,
>unless it actually masks audibility ... and there is no evidence that this is
>the case.
>
Indeed, there is no evidence that it does or that it doesn't. Why
speculate one way or another? Why not test it? Isn't that the
scientific and objective aproach?
>>One way would be to have subjectivists audition two amps (believed by
>>subjectivists to sound very differen)t using sighted tests. The amps
>
>>would be "hidden" in identical looking chasis that are only
>>differentiated by a visual designation that is unlikely to create a
>>bias. Have the "subjectivist" audition the amps as he or she normally
>>would and choose a preference. Then repeat this with two new amps
>>that are in actuality the same two amps with new designations. Repeat
>>this test until you have an adequate sample. Even though this might
>>be seen as "ambush" testing there is certainly nothing I can see that
>>would interfere with the "subjectivist" conducting listening tests
>>in pretty much the same way as they prefer to. And you can do this as
>>a controled DBT.
>
>Actually David Clark's Black Box test was an example of this.
And?
You can't leave me hanging like that!
I would love to know the details of this test that you have
referenced. I would think that in the long debate over this issue
that if anyone had conducted controled DBTs that eliminated all
objections raised by subjectivists in regards to ABX DBTs (which is
what I tried to do in the test I proposed) that such a test would be
the point of discussion in all these debates instead of ABX DBTs.
Mkuller
represented here...
>The question is, how much can we reliably know of audible difference, from
>sighted comparison?
>
>There is a difference between deciding that a component sounds good to you,
>and
>making objective claims about its sound re: another component. When I do the
>latter I try to note whether I have validated my claim in any fashion or not.
>
>And even if I sometimes fail to be rigorous in my writing about such claims,
>I
>am always ready to admit that a blind comparison might show that my sighted
>perception was illusory.
>
Now, consider that your blind comparison may be missing subtle audible cues
which were evident sighted, that had nothing to do with your bias (if any).
>The 'interest' in blind testing comes from the many, many, many claims of
>audible difference that populate audiophilia, and more importantly are
>presented as *fact*, without proper validation.
>
What you consider proper validation is not necessarily the same as what others
consider. You claim to have some science on your side. I claim that your
"science" ignores the way the brain stores and processes information when
comparing two audio components using music as the program.
Regards,
Mike
Mkuller
>>You may hear a lot of different things comparing speakers, but
>>according to your logic, they may all be figments of your imagination
>>unless you show you can hear every single difference blind. The fact
>>that there were gross differences enough that you could chosse them
>>correctly doesn't mean everything your heard sighted is valid. Or
>>does it?
>
>No, but it means that an ABX test is pointless in this case, since you
>will score 100%.
Wrong again. I'm surprised that your usually rigid logic is so
sloppy here. Scoring 100% only means there is an audible difference
that can be identified in a blind test. But how do YOU know what
difference you heard was actually audible blind? Some of them
aren't, you know...
Regards,
Mike
Nousaine
>>So far people have shown that ABX testing is sensitive to any sound
>>reproduction errors that have been shown and are known to be audible.
>>
>
>How? What has been done to determine everything that is "known to be
>audible"? I certainly hope the answer isn't DBTs. If it is you have a
>problem.
So how does one show that a difference that is "audible" on sighted evaluation
but inaudible under bias controlled listening is being masked and just not a
function of listener response bias? We'll discuss one way at the end.
>>The argument boils down to arguments that controlled listening tests haven't
>>been shown to be sensitive to reproduction differences that come from causes
>>that have so far NOT been shown to have audible consequence.
>>
>
>Well what constitutes a controled listening test is in and of itself
>another issue.
>I believe the issue isn't controled listening tests but specifically
>DBTs done in the fashion of the ABX test. Again verification that
>DBTs are omnisensitive to all human audible perception by use of
>other DBTs is fatally flawed.
Why? What you seem to be approaching here is a clever sidestep of the
subjectivist total inability to verify their claims when any kind of control
over bias is introduced.
Many people have been given opportunities to prove the claims with very simple
bias control elements (blanket over connectors, for example) and still cannot
confirm.
>>So I see this coming down to ABX testing would have to show audibility in an
>>experiment where there is no currently recognized audible cause.
>
>I disagree. I think it comes down to doing tests that do not
>interfere with the methods that subjectivists feel work in discerning
>audible differences that are claimed to be masked in ABX type DBTs.
>
>>>Is there any evidence that it doesn't?
>>
>
>>
>>How would one show that? What happens is that people often identify audible
>>consequence of nominally equivalent devices under open conditions which just
>>disappear when nothing more than forcing a choice when the answer is not
>>known
>
>>
>>in advance....even under ABX where a subject has sight of both alternatives
>>all
>>the time.
>
>I thought my test proposal was a decent idea on how to show it.
>I think if one wanted to be really thourough the best thing would be
>to do the test I suggested with amplifiers in conjuction with ABX
>DBTs, sighted tests, brain activity measurements during sighted and
>ABX DBTs and null tests on the amplifiers themselves. I think the
>data from such a combination of tests would say a lot about this
>issue
I don't think it's a real issue. But go ahead with the experiment.
>>>>>So any assumptions either way are just that, assumptions.
>>
>
>>
>>You are the only person making assumptions on this point. IMO it doesn't
>>matter
>>of bais controls affect brain activity because there is no evidence that I
>>know
>>fo that would tell us that brain activity is related to acoustical hearing
>>sensitivity.
>
>Quite the opposite. I have merly suggested that the claims of
>subjectivists that ABX DBTs mask real audible differences be
>investigated. I have made no assumptions about what the results would
>be
Stay tuned.
Dave Clark conducted an experiment with a New York audiophile society where 16
black boxes were constructed half of which contained a specific type and level
of distortion that had been shown to be audible with ABX testing.
16 subjects were mailed a black box with the instruction to decide if they had
a clean or dirty box using whatever type of in-home listening-only evaluation
technique they wanted. Results showed that these subjects were unable to
reliably determine when the distortion was present.
In 1997 I published an experiment in Audio magazine ("Flying Blind") where I
made CD-Rs that contained a level of distortion that was known to be audible
using a whole song from a well known Joan Baez song . Subjects had as long as
they wished to determine if their cd-r was clean or dirty.
The only limitation was that they were not to use a second cd player to make
direct side-by-side switched comparison but any other method was fine,
including using their personal copy of the song as a comparison.
16 subjects took up to 16 weeks to return discs. Not surprisingly analysis
showed that subjects were unable to reliably identify their disc.
Now, I invited the subject who took the longest time to return his disc
(answered incorrectly, btw) to take an ABX test using the same material. Within
a half hour the subject was able to score 100% on a 16 trial test.
IMO quick switched blind tests are the most sensitive evaluation tool. Indeed
if they have a flaw it's that they may be too sensitive to subtle differences
using specially selected program material for many decisions.
For example, they may ruthlessly feret out real differences in audibility which
may rarely occur with ordinary program material.
Now, as to elimination of subjectivist objections, the important ones have been
addressed at length. I hope you see that those were important to everybody not
just "subjectivists.' The extraneous ones, like "brain activity" simply
rearrive in different form from time to time, often as a diversion by those who
just don't like the evidentiary record.
If there are sound quality differences that cannot be found using current
objective/subjective (ABX is a subjective test) evaluative techniques It's
interesting that no subjectivist-apologist's ever provides any reasonable,
repeatable evidence of same.
Mkuller
>>
>> >Howard Ferstler fer...@attglobal.net
>> wrote:>
>
>> >Perhaps you can. However, there is only one way to validate
>> >your claims for sure: do the comparisons blind.
>
>> So you say. If the blind test interferes with the results, then it's not
>> valid. Can you understand that?
>
>We have both inadvertently cut to the chase. As I stated in
>a previous post, you appear to have set yourself up as a
>reference standard, and have done so by using sighted
>comparisons.
I have done blind comparisons and have found them to be unreliable
when the differences are subtle. Others have experienced this same
thing. My sighted comparisons have been validated by others so I
believe they are accurate.
>When a procedure (the DBT) comes along and invalidates your
>conclusions about the sound of certain products, rather than
>re-evaluate your own procedures and see if the new procedure
>offers some new insights into what is going on, you dismiss
>the new procedure and continue to hold yourself up as a
>reference standard.
See above.
>I cannot see any other way of interpreting virtually
>everything you have written about double-blind A/B testing
>in this series of threads and in a number of others as well.
>
>What can you offer us as proof of your contentions about
>double-blind comparing other than your feelings that what
>you hear is a reference standard?
I don't have proof that would be acceptable to you. Are you so rigid
in your thinking that you cannot even consider that blind testing of
audio components using music as a program could be flawed? Think
about this for a moment.
You obviously have done a lot of equipment comparisons. Are you
claiming that absolutely 100% of the audible differences you hear
sighted, are due to your biases when you cannot identify the
components blind? That's pretty black and white in a gray world, now
isn't it?
Regards,
Mike
Stewart Pinkerton
>>>mkuller wrote:>>
>>>You may hear a lot of different things comparing speakers, but
>>>according to your logic, they may all be figments of your imagination
>>>unless you show you can hear every single difference blind. The fact
>>>that there were gross differences enough that you could chosse them
>>>correctly doesn't mean everything your heard sighted is valid. Or
>>>does it?
>>
>>Stewart Pinkerton wrote:>
>>No, but it means that an ABX test is pointless in this case, since you
>>will score 100%.
>
>Wrong again. I'm surprised that your usually rigid logic is so
>sloppy here. Scoring 100% only means there is an audible difference
>that can be identified in a blind test.
By George, I think he's got it!
> But how do YOU know what
>difference you heard was actually audible blind? Some of them
>aren't, you know...
Could I have that again, please, In English? Note that we *do* try
blind tests, and when they all turn out 100%, we turn to preference.
Arny Krueger
news:akdor...@enews2.newsguy.com...
> "Arny Krueger" <ar...@hotpop.com> wrote in message
news:<akadfk$2bl$1...@bourbaki.localdomain>...
> > >
> > OTOH, it is well known that sighted evaluations involving small
> > differences interferes with the results of the test. This is a
> > scientific fact, one that has been proven many times. Why is it
that
> > the interference that a sighted evaluations cause is not
immediately
> > used as a reason to not do them?
>
> The concept of Blind A/B testing and perhaps the track record of
Blind
> A/B testing is impressive. But how practical/realistic is it for
the
> for specific components, even if they were inclined to use the
> process?
There seems to be considerable inclination to use the process when it
is easy enough to use. I think that enough test files have been
downloaded from www.pcabx.com to do over 100,000 DBTs.
>Does it not involve training to properly utilize the
> procedure?
Listener training is a good idea whether the tests are sighted or
blind. Blind testing provides considerable stimulus for listener
training, and also facilitates it. In contrast sighted listening as
usually done involves very little true training. This is because
sighted listening that involves subtle or controversial differences
can easily be controlled by the sighted aspects of the evaluation and
expectation effects.
>Or is Blind A/B testing most effectively used by
> manufacturers and researchers for development of their own
products,
> or perhaps by a few reviewers.
Blind A/B testing has been done by about 15,000 audiophiles,
engineers and audio professionals via the resources of www.pcabx.com.
The PCABX-type DBT testing methodology has been adopted by a number
of other web sites. Since I don't know what their activity levels
are, I am losing track of how many people have actually done DBTs. I
only know a minimum number based on activity at my web site, but the
number of other web sites doing similar things is increasing at what
seems to be an increasing rate.
>What reviewers regularly utilize and
> publish Blind A/B testing results.
When David Clark and Larry Greenhill were doing equipment evaluations
for Audio magazine they were regularly doing blind tests. The
technical staff of Sound & Vision do blind equipment evaluations
quite frequently, as it suits the products they evaluate.
> And how do we know that their
> testing is valid because Blind A/B testing is not so simple, or so
it
> seems (it seems like there are a lot of way to do Blind A/B testing
> incorrectly)?
It is possible to make DBTs something that a lot of audiophiles can
do for themselves with good assurance that their results are valid.
The best example I know of this is the www.pcabx.com web site. PCABX
technology has been adopted by about 8 other providers of DBT
comparators.
>How can the consumer actually utilize this information
> without actually engaging in the process?
People can share their results. There seems to be quite a bit of
interest in finding out what kind of results other listeners are
obtaining. I am working on making this more reliable, automated,
verifiable, and formalized.
>How can the cross section of
> participants of this board, for example, effectively use Blind A/B
> testing?
I suspect that many participants of this newsgroup have visited
www.pcabx.com and done some DBTs.
>A couple of years ago I wanted to compare two amps I was
> considering, the Musical Fidelity Nu Vista 300 and a Spectural
model,
> how could I have practically utilized Blind A/B to arrive at a
> decision?
PCABX procedures can make comparisons between a range of amplifiers
in any combination available to the general public.
>Is Blind A/B testing meant for participation by the cross
> section of audiophiles.'
Yes, I think so.
BEAR
>
<snippo'd>
>
>
> In 1997 I published an experiment in Audio magazine ("Flying Blind") where I
> made CD-Rs that contained a level of distortion that was known to be audible
> using a whole song from a well known Joan Baez song . Subjects had as long as
> they wished to determine if their cd-r was clean or dirty.
>
> The only limitation was that they were not to use a second cd player to make
> direct side-by-side switched comparison but any other method was fine,
> including using their personal copy of the song as a comparison.
>
> 16 subjects took up to 16 weeks to return discs. Not surprisingly analysis
> showed that subjects were unable to reliably identify their disc.
>
> Now, I invited the subject who took the longest time to return his disc
> (answered incorrectly, btw) to take an ABX test using the same material. Within
> a half hour the subject was able to score 100% on a 16 trial test.
>
> IMO quick switched blind tests are the most sensitive evaluation tool.
Actually, they are "a sensitive evaluation tool" FOR certain things.
Let's not get carried away here by declaring *victory* for ABX.
For example ABX is of little value for hearing the effect of a small
tweak to a crossover. Simple pink noise is far more effective without
any "X" or DBT.
> Indeed
> if they have a flaw it's that they may be too sensitive to subtle differences
> using specially selected program material for many decisions.
The construction of this sentence makes it hard for me to understand
the intent or meaning - perhaps you could restate this with a clearer
statement?
>
>
> For example, they may ruthlessly feret out real differences in audibility which
> may rarely occur with ordinary program material.
What is a "real difference in audibility"? Especially those that
"rarely occur with ordinary program material"?? What sort of program
material are we talking about?
>
>
> Now, as to elimination of subjectivist objections, the important ones have been
> addressed at length. I hope you see that those were important to everybody not
> just "subjectivists.' The extraneous ones, like "brain activity" simply
> rearrive in different form from time to time, often as a diversion by those who
> just don't like the evidentiary record.
Let's take the term "brain activity" and translate that as a short
hand for something like 'effort made by the brain to identify and
interpret the raw sound data it is presented with.' Since I did not
use the term "brain activity" myself, and do not chose it as a term
that I think is appropriately precise for this sort of discusssion,
I've just taken the liberty of adjusting it to something that I
belive makes sense in the context under discussion.
Let's review this idea a bit.
Since sound is a real time event, it must be processed as received.
While that may include a degree of 'buffering' there is a finite
'buffer space' that people have. Indeed, sound in the context of
speech or music is *not* interpreted by the brain either in serial
bits, serial chunks but rather is 'strung together' more like a
spline that has the end falling off and the front always being added
to. Along those lines, what goes on in creating this mental 'string'
or 'spline' is not only unclear, but it is almost certainly quite
*different* for different people. The easiest example is a trained
musician with perfect pitch and physiologically perfect hearing vs. a
person untrained in any way, with limited experience and perhaps a
physiological defect to add to the mix.
Clearly, there are some things that people in general "hear" more
easily than other things - sort of built in genetic coding and basic
training. Beyond that there are wide ranges of subtle and not so
subtle differences - most of which are the result of some sort of
*learned* internal processes.
So, no matter how you slice it, when you hear something you are first
making an identification, then interpreting it.
It is fairly clear and demonstrable that short bursts of sound are
NOT interpreted the same way as LONG periods of sound. The degree of
this matters, yes.
Perhaps the ONLY difference between what the subjectivsts are
"hearing" and what the "objectivists" are claiming is not so much
this vaunted difference that can be QUANTIFIED, as much as it is an
*ease* with which the brain can identify, process and interpret what
sound it is being presented with.
So, in the end there is NO difference in ABX presentations of a given
sound, IF the brain sucessfully processes and *decodes* the sound!
However, there *is* a meta-difference in terms of the speed and ease
of that work being done by the brain. I submit that this is what the
subjectivist FEELS (he can only *feel* that the brain working to
decode and process) since it is not an explicit and overt thing, and
so proclaims the existance of a "difference."
>
>
> If there are sound quality differences that cannot be found using current
> objective/subjective (ABX is a subjective test) evaluative techniques It's
> interesting that no subjectivist-apologist's ever provides any reasonable,
> repeatable evidence of same.
Not so sure on this one... but no one seemed to follow up on the two
examples of brain + processing = interpretation of sound that I made
in my first post to this thread. The one that added in the " - not so
simple." Per the above text, too.
And the term "subjectivist-apologist" is a pejorative that is
unecessary coming from a person whose published articles in several
magazines are (as previously discussed on this forum) at best highly
questionable.
_-_-bear
--
_-_-bearlabs
http://www.bearlabsUSA.com
- Silver Lightning Interconnects -
Bob Marcus
> > Gary Eickmeier geic...@tampabay.rr.com
> wrote:>
> >This sure was a long post, and this certainly was a long thread, for
> >something so irrelevant.
> >
>
> If that's so, why did you post so many times?
>
> >Why all the interest in blind testing? Has someone been pushing it for
> >audiophiles?
>
> Stewart Pinkerton claims it's the only way to know what you are
> a start.
>
> >Why does this topic keep coming up so often here?
>
> It takes two sides for an arguement to occur...
Yes, but it only takes one side to *start* the argument. Who's been
starting all these threads?
bob
Arny Krueger
> What you consider proper validation is not necessarily the same as
> what others consider.
Some people are easy to please. You can fool some of the people all
the time and all the people some of the time, I believe a wise man
said.
>You claim to have some science on your side.
While others ignore what science says about human perceptions.
> I claim that your "science" ignores the way the brain stores and
> processes information when comparing two audio components using
> music as the program.
I claim that the moon is made out of green cheese. I believe that
both statements are called "proof by assertion".
How convinced were the followers of that evangelist who had his wife
send him personal information about visitors to the congregation over
a earpiece. I'll bet that those people claimed that they had seen
true and genuine miracles.
Arny Krueger
news:ake88...@enews2.newsguy.com...
>
> For example ABX is of little value for hearing the effect of a small
> tweak to a crossover. Simple pink noise is far more effective without
> any "X" or DBT.
Let's say that we were presented with a so-called "multiple choice
test" with only one choice per question. This is like an ABX test
without any X's. In plain words, an ABX test without X's isn't a
test.
Steve Sullivan
> >Stev...@aol.com wrote:>
> >er..he's not alone , you know.
> >
> >He has science on his side too
> >
> Uh, you mean he has you on his side with your understanding of the science
> represented here...
Nope, that's not what I mean, sorry.
>
> >The question is, how much can we reliably know of audible difference, from
> >sighted comparison?
> >
> >There is a difference between deciding that a component sounds good to you,
> >and
> >making objective claims about its sound re: another component. When I do the
> >latter I try to note whether I have validated my claim in any fashion or not.
> >
> >And even if I sometimes fail to be rigorous in my writing about such claims,
> >I
> >am always ready to admit that a blind comparison might show that my sighted
> >perception was illusory.
> >
> Now, consider that your blind comparison may be missing subtle audible cues
> which were evident sighted, that had nothing to do with your bias (if any).
>
If you or I reported that we heard something sighted , that disappeared blind,
there's already a good scientific explanation available for what happened. Your
hypothesis, on the other hand -- that 'subtle audible clues', rather than sighted
bias, are what disappear when the 'sightedness' is controlled for -- has as yet
no scientific support. Feel free to provide some, if you can.
>
> >The 'interest' in blind testing comes from the many, many, many claims of
> >audible difference that populate audiophilia, and more importantly are
> >presented as *fact*, without proper validation.
> >
> What you consider proper validation is not necessarily the same as what others
> consider.
It is, however, what proper science would require.
> You claim to have some science on your side. I claim that your
> "science" ignores the way the brain stores and processes information when
> comparing two audio components using music as the program.
Yes, I've seen you claim that a few times. Yet I know for a fact that 'my'
science is very interested in the way the brain stores and processes
information. So,where's your data? I presume this data provides evidence that
knowledge of what component is playing, increases the sensitivity to real audible
differences.
"Cos that's what you seem to be implying.
Jamie Benchimol
No, all you're saying here is that an A/B test isn't a "multiple
choice test". You seem to think only one kind of test is valid, to
obtain objective results. Namely the "ABX" test that you've trumpeted
for so many years. But the relevance of this sort of misapplied
methodology to real-world audio evaluations has been refuted countless
times over to you, in as many ways. Why are you still defending it,
then? It is intellectually dishonest of you to do so at this point.
Bruce Abrams
idea that ABX testing introduces some psychological stress or other factors
that obfuscate what might be "subtle audible differences." Along of the
lines of Norman Strong's hypothetical question, to satisfy those who
subscribe to the previously stated thinking, why not eliminate the "ABX
Stress" and remove sighted bias, by simply placing the components in
question into two boxes, one black and one white.
You can now look at the amps (let's use amps as a simple example) all you
want, and switch to your heart's content. As long as you don't know which
is in which box, you've at least removed an entire range of preconceptions.
(As long as agreed in your own mind to keep the volume below a certain
level, imagine how much this could be with a 6 WPC SET and a Krell FBP 600!)
So now you can do a long term listening comparison simply not knowing what
amps you're listening to. Once you've taken notes, listened in as relaxed a
manner as possible and generally feel real comfortable with your
understanding of the two amps, you might even find an ABX test fun. And if
you really can't possibly bear the thought of ABX'ing the two boxes, you
could introduce a fun twist into the scenario.
Have a friend switch the contents of the boxes periodically without telling
you, and then see if your listening notes correlate with the switches.
"Mkuller" <mku...@aol.com> wrote in message
news:ake11...@enews3.newsguy.com...
Howard Ferstler
> >> >Howard Ferstler fer...@attglobal.net
> >> wrote:>
> >What can you offer us as proof of your contentions about
> >double-blind comparing other than your feelings that what
> >you hear is a reference standard?
> I don't have proof that would be acceptable to you. Are you so rigid
> in your thinking that you cannot even consider that blind testing of
> audio components using music as a program could be flawed? Think
> about this for a moment.
I certainly have. The potential flaws are relatively straightforward:
(1) a failure to properly level match on all channels, (2) a failure
to use quick switching, which would add needless complexity to the
switchovers, (3) a failure to let the participant run the switch box
himself, and (4) a failure to let the participant use whatever program
source material he chooses.
On the other hand, a screwed-up sighted comparison would include those
four and would also include a fifth that would be monumental: an
inability to guarantee that the participant's preconceptions would not
skew the results.
> You obviously have done a lot of equipment comparisons. Are you
> claiming that absolutely 100% of the audible differences you hear
> sighted, are due to your biases when you cannot identify the
> components blind?
Actually, I have never heard differences between decent amps and
wires, whether I compared blind or sighted. Since it is unlikely that
those items I have compared have all had identical distortions, I
assume they had none that were audible. Yes, I compare sighted with
amps and wires fairly often. However, I do not make any extraordinary
claims about being able to hear differences. And even if differences
were there, they would be so small as to be undetected with
straightforward, single-presentation listening.
With speakers, all bets are off. They indeed can sound different for a
multitude of reasons (possibly the most notable are differences in
broad-bandwidth radiation patterns), and I report on said differences
in my product reviews. Those reviews combine objective and subjective
techniques, and I make no attempt to hide that fact.
Note that this is just for me. If people trust my results, then they
can move on and act upon them. If they do not, I suggest they do
similar tests themselves. If they hear differences sighted, I suggest
they do the procedure again, blind, just to make sure.
After all, they are the only person they need to satisfy.
Howard Ferstler
ludovic mirabel
All that you say in praise of ABX type testing for component
differences is founded on a premise:
When subject to ABXing all of us hear all that they can hear in normal
listening.
This premise is not proven, It is an "obvious truth" to you but not to
me. In fact there is considerable body of evidence in favour of the
contrary premise- under Abx many don't hear everything that they would
otherwise. (for a fragment of it see my reply to Mr. Eickmayer on
Aug.23)
You get rid of the crudest kind of bias. the only one that many here
can understand: marketing hype. The question is : what else do you get
rid of?
The unproven premise is sometimes modified to: if you can't hear it
ABXing it wasn't worth hearing in the first place.
This assumes that we all have the same scale of "worth"- a manifest
nonsense especially when coming from self-nominated spokesmen for
"science"
Till you can prove your hypotheses no amount of extolling the virtues
of your test, its manifest beauty, simplicity, and good sense will
make any impact on those for whom it doesn't work.
Ludovic Mirabel
_______________
Bruce Abrams
impossible to remove biases that influence the test. Therefore, the test
results don't represent data at all, merely an opinion. "I think that amp
sounds ... than that other amp." Your contention that DBT or ABX testing is
meaningless because it interferes with your ability to listen means that you
have to go back to sighted tests, which we already agreed were flawed (ie:
they suffer at least from the influences of marketing hype.)
Once again...ABX isn't designed to determine preference, merely difference.
Is it possible that you could listen to two components sighted for several
weeks so that you really knew their unique sonic signatures, and then have
someone switch them and you wouldn't notice? I suspect that if the
components in question are competently designed amplifiers, the answer is
yes. More to the point, how would you ever know if all you've ever done is
seen what you were listening to?
So what, you might say. I'm happy in my state of ignorant bliss. I know
what I hear and no empirical evidence proving otherwise will make think
otherwise.
I've made the following point before and I'll make it again. If you're
convinced (by a salesman, magazine article, ad, etc.) only through sighted
testing, that $1,000 of speaker cable are necessary for you to get the best
out of your system, and your budget for speakers, amp & cables is $5,000,
you might spend a grand on cables, $1,500 for an amp and $2,500 for
speakers. This might get you a nice system, but don't you think that extra
grand would be considerably better spent on speakers?
How 'bout Mpingo Disks, or Shakti stones, or.....If those products died
quickly because nobody could provide data proving their effect, many people
would have had either much better systems, or much more music to listen to.
That's why it's important.
"ludovic mirabel" <elm...@pacificcoast.net> wrote in message
news:akgatk$18o$1...@bourbaki.localdomain...
BEAR
> "BEAR" <bear...@netzero.net> wrote in message
> news:ake88...@enews2.newsguy.com...
> >
> > For example ABX is of little value for hearing the effect of a small
> > tweak to a crossover. Simple pink noise is far more effective without
> > any "X" or DBT.
Let me restate that, in more proper terms... simple pink noise is
sufficiently and substantially more effective in the above case as to
obviate the need for any "X" state when compared to testing the same
change(s) using normal program material. Indeed, the same change(s)
tested using ABX would likely be inaudible and indetectable, in my
estimation and based upon my experience.
>
>
> Let's say that we were presented with a so-called "multiple choice
> test" with only one choice per question.
By definition a multiple choice test has more than two choices, but
certainly it is necessary to have at least two, so I have no idea
what you are thinking of.
> This is like an ABX test
> without any X's. In plain words, an ABX test without X's isn't a
> test.
Au contraire, mon ami, Arny.
Tests are not defined as being only ABX.
It is true that an ABX test without an X is not an ABX test.
Also, not all tests have "answers." Some tests have *results*. A
simple example is an FFT of a device - the outcome of the test is a
result - one that then has to be interpreted - it is neither right
nor wrong. But it is a test.
But this is ad absurdum, since the main point I made obviates this
discussion of minutia. Mais Oui!
Nousaine
I'd agree but would like to also suggest that controlled listening,
like any good evaluative test, doesn't need to be actually used by
any interested party to be of utility.
For example, a consumer doesn't actually need to perform mileage
tests on all the vehicles he intends to consider for purchase . The
EPA results, while perhaps not precise for any particular driver,
will provide valuable information to guide purchase. Same with top
speed, 0-60, skid pad results... any individual doesn't actually need
a skid pad to benefit from their results.
As for amplifier comparisons one doesn't necessarily need to conduct
a blind test of those two specific amplifiers to benefit from the
knowledge that no one to date has ever been able to identify
nominally competent amplifiers under acoustical conditions in typical
living room conditions.
Arny Krueger
news:akfep9$3b7$1...@bourbaki.localdomain
> "Arny Krueger" <ar...@hotpop.com> wrote in message
> news:<akeds...@enews3.newsguy.com>...
>> "BEAR" <bear...@netzero.net> wrote in message
>> news:ake88...@enews2.newsguy.com...
>>>
>>> For example ABX is of little value for hearing the effect of a
>>> small tweak to a crossover. Simple pink noise is far more
>>> effective without any "X" or DBT.
>>
>> Let's say that we were presented with a so-called "multiple
>> choice test" with only one choice per question. This is like an
>> ABX test without any X's. In plain words, an ABX test without
>> X's isn't a test.
> No, all you're saying here is that an A/B test isn't a "multiple
> choice test".
A sighted A/B test is not an adequate test when subtle or
controversial issues are being studied.
>You seem to think only one kind of test is valid, to
> obtain objective results. Namely the "ABX" test that you've
> trumpeted for so many years.
Not at all. There are many kinds of DBT that are valid, including
ABX, ABC/hr, same/different, various flavors of triangle test, etc.
etc., etc.
Furthermore, the goal of most of the listening tests that I propose
and support is not an objective result, but simply a reliable result
that is based on just listening.
>But the relevance of this sort of
> misapplied methodology to real-world audio evaluations has been
> refuted countless times over to you, in as many ways.
I've shown that this last paragraph is an argument against a straw
man because I clearly don't think that the ABX test is the only valid
test. Most people here know that I even think that sighted tests can
be valid when the audible differences are unmistakable, such in
loudspeaker sound quality evaluations.
> Why are you still defending it, then?
I'm not.
>It is intellectually dishonest of you to do so at this point.
Good thing that I'm not defending it, right?
;-)
Nousaine
Actually I agree that pink noise is probably the single best overall
evaluative signal possible because it contains every sound ever heard
by a human in history all at once. As such, it's one great signal to
use with or without bias controls.
It's not perfect because it may not uncover some dynamic problems
but....
The idea that a given signal eliminates the need for listener bias
controls is simply a diversion. But pink noise has the same problem
that ABX has .... it can be too sensitive, exposing artifacts that
may never be heard with music.
Emerson Wood
> emers...@yahoo.com (Emerson Wood) wrote:
>
> >fer...@attglobal.net (Howard Ferstler) wrote in message
> >news:<ak62j...@enews3.newsguy.com>...
> >> emers...@yahoo.com (Emerson Wood) wrote in message
> news:<ak31a...@enews2.newsguy.com>...> news:<ajr4h...@enews2.newsguy.com>...
>
> >> > > How would you know this is true? You know what is playing,
> >> > > and so you could easily be fooled by your own
> >> > > preconceptions. It is fatuous to claim that because
> >> > > differences disappear when listening blind those differences
> >> > > must somehow be obscured by the blind-comparison protocol.
> >> > >
> >> > > Most people would say just the opposite: sighted comparisons
> >> > > introduce potential prejudices that blind comparisons will
> >> > > not allow to be introduced.
>
> >> > Right--blind comparisons just have their own prejudices.
> >>
> >> And what might those be? The fact is that no matter what you might
> >> think about blind comparisons screwing up one's perceptive
> >> capabilities, at least the results cannot be fabricated. You either
> >> can or cannot hear differences, and there is no way to pick out your
> >> favorite amp or set of wires because they happen to be your favorite.
> >> You have to judge by sound alone. You cannot be prejudiced one way or
> >> the other, because you have no idea which component is playing during
> >> the blind phase of the test.
> >
> >In answer to your question, I listed some of the mechanisms
> >in my post which you snipped. The prejudices would be to bias
> >the result either (a) toward "no difference" or (b) toward hearing
> >a marked difference that is relatively irrelevant to *music*
> >listening.
>
> Actually the normal human bias is to report different even when given
> identical alternatives. You can see this is the Stereophile tests
> where subjects would answer 'different' about twice as often as
> 'same.' . Also check "Can You Trust Your Ears?" AES Preprint and
> Stereo Review where it can be seen that subjects will report
> different over 3/4 of the time when given two identical sound
> presentations.
> This is one of the prejudices that actually exists and can be shown
> to exist.
But *in a DBT*? A test such as ABX is designed so that those
impressions statistically cancel out. We're talking about the
prejudices of the *whole system*, the human, equipment, and DBT
protocal taken as a whole system.
I have no problem accepting the existence of the mechanism
you describe, but I do suspect that the observations of it
are distorting the results. It's like trying to discover the
natural behavior of a great white shark while confining it to
a tank. The idea of juxtaposing two sounds and asking for
differences, in the usual way of doing so, right away takes
a person out of a natural music-listening context.
-Emerson
S888Wheel
>>another issue.
>>I believe the issue isn't controled listening tests but specifically
>>DBTs done in the fashion of the ABX test. Again verification that
>>DBTs are omnisensitive to all human audible perception by use of
>>other DBTs is fatally flawed.
>
>
>Why? What you seem to be approaching here is a clever sidestep of the
>subjectivist total inability to verify their claims when any kind of
>control over bias is introduced.
>
Why? I find I am having a tough time illistrating why but I will try.
Maybe if I put the shoe on the other foot it will help illustrate why
verification of the effectiveness of a test by repeating the test is
a fatally flawed method. Say some audiophile does sighted tests of a
new cable and declares it is a night and day difference. He verifies
it by having all his audiophile friends do the same sighted tests.
They all reach the same conclusion. Has his test been verified? No.
If the test is inherently flawed you cannot find that flaw by
repeating the same type of test .My critisizm is not meant to be a
clever side step. Many people seem to have issues with ABX DBTs. You
cannot convince any of those people by claiming previous DBTs have
confirmed current DBTs. It simply becomes selfserving. Alternative
verification isn't always easy to come by but it is powerful evidence
when you can get it.
>
>Many people have been given opportunities to prove the claims with
>very simple bias control elements (blanket over connectors, for
>example) and still cannot confirm.
I think many noble attempts have been made to avoid any potential
shortcommings of DBTs. Unfortunately none so far that I have seen
have addressed the assertion being debated in this thread.
>>I thought my test proposal was a decent idea on how to show it.
>>I think if one wanted to be really thourough the best thing would be
>>to do the test I suggested with amplifiers in conjuction with ABX
>
>>DBTs, sighted tests, brain activity measurements during sighted and
>>ABX DBTs and null tests on the amplifiers themselves. I think the
>>data from such a combination of tests would say a lot about this
>>issue
>
>I don't think it's a real issue. But go ahead with the experiment.
>
Alas as interesting as this thread has been I must confess I have
neither the time nor resources to conduct such tests. This is not my
field and I suspect I am not really qualified to do these sorts of
tests anyways.Besides I really need to devote any R&D time to my own
proffession. I understand why you or anyone else who does not think
this is an issue would not bother to do such tests either.But I am
afraid the assertion will linger until it is addressed with proper
tests.
>Dave Clark conducted an experiment with a New York audiophile society
>where 16 black boxes were constructed half of which contained a
>specific type and level of distortion that had been shown to be
>audible with ABX testing.
>16 subjects were mailed a black box with the instruction to decide if
>they had a clean or dirty box using whatever type of in-home
>listening-only evaluation technique they wanted. Results showed that
>these subjects were unable to reliably determine when the distortion
>was present.
An interesting test. Unfortunately it is lacking in certain controls(
the test does not eliminate issues of system resolution or hearing
acuity of each individual or potential mistakes in each persons
methods of listening) and I have a problem with what amounts to
single samples of each individual. Averaging the results of many
testees who are left to their own only indicates what the average
person can or cannot do under different conditions. This test also
unfortunately still does not address the assertion that
identification under blind conditions is inherently masking otherwise
audible differences.The subjects of this test still had to choose
under DBT conditions which they were aware of. Also since I don't
know the actual test results only the short analysis of them it is
hard to comment. I would be curious to know how many incorrect IDs of
dirty box were made and how many incorrect IDs of clean boxes were
made.
>In 1997 I published an experiment in Audio magazine ("Flying Blind") where I
>made CD-Rs that contained a level of distortion that was known to be audible
>using a whole song from a well known Joan Baez song . Subjects had as long as
>they wished to determine if their cd-r was clean or dirty.
>
>
>The only limitation was that they were not to use a second cd player to make
>direct side-by-side switched comparison but any other method was fine,
>including using their personal copy of the song as a comparison.
>
>
>16 subjects took up to 16 weeks to return discs. Not surprisingly analysis
>showed that subjects were unable to reliably identify their disc.
>
>
>Now, I invited the subject who took the longest time to return his
>disc (answered incorrectly, btw) to take an ABX test using the same
>material. Within a half hour the subject was able to score 100% on a
>16 trial test.
Another interesting test. The obvious question for me to you is how
many incorrect answers were that the CD-R was dirty and how many
incorrect answers that the CD-R was clean were made. I ask this
because anyone who happens to have a CD of that title that was
mastered differently and could possibly hear that difference and give
a legitimate false positive. And anyone who's system isn't capable of
rendering the added distortion or who's hearing isn't acute enough
could give a legitmate false negative. I'm affraid that single
samples of many different people under many different conditions
don't really tell us everything about what is or is not audible. I
thank you for your post. It was very interesting.
>IMO quick switched blind tests are the most sensitive evaluation
>tool. Indeed if they have a flaw it's that they may be too sensitive
>to subtle differences using specially selected program material for
>many decisions.
>For example, they may ruthlessly feret out real differences in
>audibility which may rarely occur with ordinary program material.
If nothing else you have made a compelling arguement that this is a
very usefull test even if the assertion that is being debated has not
been resolved one way or the other.
>Now, as to elimination of subjectivist objections, the important ones
>have been addressed at length. I hope you see that those were
>important to everybody not just "subjectivists.' The extraneous ones,
>like "brain activity" simply rearrive in different form from time to
>time, often as a diversion by those who just don't like the
>evidentiary record.
I have no opinion one way or the other as to whether the current set
of objections held by subjectivists are extraneous or legitimate. I
understand if one feels that the investigation of these claims are
not worth the effort. On the other hand it does seem to me that a
fair amount of effort has gone into the arguement. I have a question
though, "the evidentiary record" you alude to, where can I find it. I
often find the thing most lacking in these debates is the actual data
of DBTs. I can see this data as a potential mind field filled with
poorly conducted tests. Are their any websites with data of DBTs
which were done proffessionally and document the conditions and
equipment as well as the raw data? This would be interesting to me.
>If there are sound quality differences that cannot be found using
>current objective/subjective (ABX is a subjective test) evaluative
>techniques It's interesting that no subjectivist-apologist's ever
>provides any reasonable, repeatable evidence of same.
Perhaps it is the subjectivists who should conduct my proposed test.
Do you think the test I proposed is a fair and legitimate controled
test?
Thank you again for your responses. I apologize for all spelling
errors.
Jamie Benchimol
> "Mkuller" <mku...@aol.com> wrote in message
> news:akdu4p$eu8$1...@bourbaki.localdomain...
>
> > What you consider proper validation is not necessarily the same as
> > what others consider.
>
> Some people are easy to please. You can fool some of the people all
> the time and all the people some of the time, I believe a wise man
> said.
I think that with your senseless "ABX" paradigm, a misapplication of
scientific principles, you are trying to do both.
> > I claim that your "science" ignores the way the brain stores and
> > processes information when comparing two audio components using
> > music as the program.
I agree with Mkuller's statement.
> I claim that the moon is made out of green cheese.
Can you prove this Arny?
> I believe that both statements are called "proof by assertion".
Then it is up to you to prove the claim wrong. You have always had a
very weak understanding of science Arny, because you do not have a
scientific background. This is one of the key reasons for why your ABX
methodology is misapplied to audio evaluation. Because you do not have
a deep understanding of how the human brain might work in such
situations, you make assumptions based on a lack of scientific data.
Which in itself, is very UNscientific of you.
But then, I don't think you are at all interested in discovering what
is really going on during an ABX test, because you have a vested
interested in keeping the "myth of ABX" alive, even though it was
largely dismissed back in the 70's. Your interest of course, comes
from the fact that you, and a couple of your friends, have in recent
years put out "ABX software". As you have a vested interest in the
business end of ABX, I think that you should be compelled to note your
commercial affiliations to the ABX venture in your sig if you are
going to discuss ABX. TIA.
> How convinced were the followers of that evangelist who had his wife
> send him personal information about visitors to the congregation over
> a earpiece. I'll bet that those people claimed that they had seen
> true and genuine miracles.
To answer your question, I'd say they were about as convinced as
anyone who buys into your ABX religion, and thinks it has a place in
the real world of evaluating audio components. Fair enough?
C. Leeds
> Training is irrelevant.
Your belief conflicts with just about everyone who's ever been
seriously involved in ABX testing. In this very thread. Arny wrote:
> > I offer training because listening is a skill that can be developed
> > for greater sensitivity and reliability, whether sighted or blind.
Howard Ferstler
The primary advantage of the ABX protocol is that it combines a
sighted comparison (A and B are both known) with a blind comparison (X
is unknown, and during any of the multiple trials it may be either A
or B).
This allows the participant to compare the known A and B components as
much as they like, in order to determine their respective sonic
characteristics. Then, when they switch to X (and they can go back and
forth between X and A and B as much as they want), all they need to do
is determine if X has the characteristics of A or those of B. That is
one trial. It is proper to do a lot more trials, of course, and X may
be either A or B with any of them. Only the ABX device knows, and the
participant's score can be determined at any time during the series of
comparisons.
It is hard for me to fathom a more straightforward way to compare
components for subjective performance.
Howard Ferstler
Arny Krueger
> You have always
> had a very weak understanding of science Arny, because you do not
> have a scientific background.
Since we are comparing resumes here, why not tell us about yours, Mr.
Benchimol.
In short, I'm an engineer with a BS degree. I did most of the work
required for a MS, as well. I've worked in industry (various
positions) since I left school due to my wife's pregnancy in 1973 or
so.
> This is one of the key reasons for
> why your ABX methodology is misapplied to audio evaluation.
ABX has been credentialed by the Audio Engineering Society (AES) for
audio applications since the (refereed) publication of Clark, David
L., "High-Resolution Subjective Testing Using a Double-Blind
Comparator", Journal of the Audio Engineering Society, Vol. 30 No. 5,
May 1982, pp. 330-338.
That was over 20 years ago. The article has never been discredited,
and ABX listening tests are used in industry to this day.
Now here's the question you need to answer, Mr. Benchimol. What
publication in a refereed scientific journal counters David Clark's
JAES article?
> Because you do not have a deep understanding of how the human
> brain might work in such situations, you make assumptions based
> on a lack of scientific data. Which in itself, is very
> UNscientific of you.
You seem to have an uncanny ability to read my mind, Mr. Benchimol.
> But then, I don't think you are at all interested in discovering
> what is really going on during an ABX test, because you have a
> vested interested in keeping the "myth of ABX" alive, even though
> it was largely dismissed back in the 70's.
Publications please?
>Your interest of
> course, comes from the fact that you, and a couple of your
> friends, have in recent years put out "ABX software". As you have
> a vested interest in the business end of ABX, I think that you
> should be compelled to note your commercial affiliations to the
> ABX venture in your sig if you are going to discuss ABX. TIA.
My affiliation with the www.pcabx.com web site is well known. Let me
take this opportunity to refer you to http://www.pcabx.com/inidex.htm
which offers the following ABC/hr (in some sense competitive with
ABX) software:
"An ABC/Hr comparator can be found at Darryl Miyaguchi's web site at
http://ff123.net/abchr/abchr.html "
>> How convinced were the followers of that evangelist who had his
>> wife send him personal information about visitors to the
>> congregation over a earpiece. I'll bet that those people claimed
>> that they had seen true and genuine miracles.
> To answer your question, I'd say they were about as convinced as
> anyone who buys into your ABX religion, and thinks it has a place
> in the real world of evaluating audio components. Fair enough?
I think its quite an unusual religion that has received as much
favorable treatment by respected industry and scientific journals as
ABX and/or PCABX. You can find the references at
http://www.pcabx.com/index.htm and
http://users.htdconnect.com/~djcarlst/abx_peri.htm .
ludovic mirabel
> > I see no
> >>>bigger deal with a blind test than with a sighted test, but obviously
> >>>those who are sweating the status of their golden ears or the
> >>>performance of a favorite set of wires or amp will probably have a
> >>>rather unsettled state of mind during the test. Life is tough.
> >>
SNIP. See below.
As you're obviously om the side of the angels it surprises me that
you are willing to consider seriously abstruse, contradictory ideas of
how the brain works based on pure speculation without any factual,
experimental backing. It is all too reminiscent of "noxious miasmas"
and "four humours" that used to explain disease before bacteriology
and physiology existed.
In the meantime facts that stare one in the eye demanding explanation
are shoved aside:
1) People vary. Their minds process musical signals according to their
unique processors formed by an inextricable pattern of their nature
and environmental nurture. Any "test" for differences between music
reproduction by components will have responses as subjective as the
people being tested ie it will be as subjective as can be. Evidence:
widely divergent individual results of any such ABX testing.
2) The particular testing method namely ABX switching has not been
PROVEN not to interfere with reception of musical signals for at least
some of the individuals tested,
In fact there is abundance of facts that suggest that it does. See my
241st. Aug. reply to Mr. Eickmayer (message 11) in the "Why blind
ABX...etc" thread. See selecting and training requirements
in the psychometric research. See Krueger's ABX training offerings.
3) Till such proof exists the results of any ABX testing must be
considered valid only for ABX able individuals on their own individual
basis. And with reservations at that since it cannot be disproved that
even they might have done better without it.
Ludovic Mirabel
>
> >>This reminds me of people who wonder why people with Terets simply
> >>just don't do the things they do. The human mind is a complex system.
> >>It is a well known fact that seemingly related tasks such as writing
>
>
> >>language and speaking it use very different parts of the brain. You
> >>can't just will it away.
> >
>
> >
> >This is precisely the problem with open evaluation for sound quality
> >differences. There are human response bias mechanisms (level/quality
> >confusion, pro-choice decision tendency are two) that cannot be
> >willed away.
>
> The question at hand is whether or not this is a problem with ABX
> DBTs. Critiques of sighted tests, while certainly valid, don't
> really address the question at hand.
>
> >>If the task of doing ABX DBTs is masking
> >>something in our perceptions it is a problem. That this is a problem
> >>with ABX listening tests hasn't been proven but has it been
> >>disproven?
>
> >
> >
> >There is no evidence that controls for human listening response bias
> >affect anything except non-sonic bias mechanisms.
> >
>
> Is there any evidence that it doesn't?
>
> >>So any assumptions either way are just that, assumptions.
> >>Unless there have been some tests I don't know about that addressed
> >>this issue. It would be very easy to test whether or not the brain
> >>changes gears in ABX DBTs although I suspect it would be expensive.
>
> >
> >Even if brain activity were found to change that is not evidence that
> >any sonic factors would have been masked.
>
> I agree but what this test could offer is evidence in regards to the
> propostion that ABX DBTs changes the "state of mind" of the testee.
> If such tests showed a significant change in brain activity then the
> propostion is not so outrageous, if such tests showed little or no
> measurable changes in brain activity then the proposition looses
> credibilty.
>
> The answer I think is obvious. Do controled blind tests of amplifier
> and/or wire comparisons that allow "subjectivists" to do sighted
> listening tests in the same manner as they do when they claim to hear
> differences that "objectivists" feel are unreliable do to the lack of
> propper controls.