A Brief History of CD DBTs

[nab...@hotmail.com]

For anyone who has slogged through the current thread on CD sound and wonders where the evidence really lies, here’s a brief summary. There have been numerous published DBTs of CD players and DACs, and the bottom line of the results agrees with the accepted theory of psychoacoustics experts: there is no audible difference among conventionally designed products. The very rare differences that have been found can be explained by the unusual designs in question.

Published DBTs begin with the article cited in the other thread:

Masters, Ian G. and Clark, D. L., "Do All CD Players Sound the Same?", Stereo Review, pp.50-57 (January 1986)

If memory serves, they did find one CD player that was audibly distinguishable from the others. I believe it was an early 14-bit model from Philips.

Two later tests also appeared in SR:

Pohlmann, Ken C., "6 Top CD Players: Can You Hear the Difference?", Stereo Review, pp.76-84 (December 1988)

Pohlmann, Ken C., "The New CD Players, Can You Hear the Difference?", Stereo Review, pp.60-67 (October 1990)

Both tests found no differences among players.

The Sensible Sound did a two-part report on another test:

CD Player Comparison, The Sensible Sound, # 75, Jun/Jul 1999.

CD Player Comparison, The Sensible Sound, # 74, Apr/May 1999.

My understanding is that they did not identify the actual players being tested, except for the cheapest one, which was a sub-$100 carousel model. Again, no differences were found.

A group in Spain has posted results of numerous tests it has done. A full list of tests is here, unfortunately in Spanish:

http://www.matrixhifi.com/marco.htm

(click on Pruebas Ciegas to see the list)

Most of their tests found no audible differences. (See, for example, their comparison of a Benchmark DAC to a Pioneer DVD player.) Devices that did sound different:

1) a non-oversampling DAC
2) a device with a tubed output stage
3) a portable Sony Discman, connected via its headphone output

Two further points:

1) No quantity of DBTs can prove a negative. But believers in CD/DAC sound can cite no comparable empirical evidence whatsoever for their position.

2) Psychoacoustics researchers have reached the same conclusion via other means. Here’s a standard textbook in the field:

Moore, BCJ. An Introduction to the Psychology of Hearing, Fourth Edition. San Diego: Academic Press, 1997.

And here’s what Dr. Moore had to say about the issue:

“CD and DAT players generally have a specification which is far better than that of other components in a hi-fi system, especially cassette decks and loudspeakers. Essentially, the output signal which they provide is indistinguishable from that which would be obtained from the master tape produced by the recording studio (studio recordings are now usually digital recordings). Thus, provided a CD or DAT player is working according to specification, it will produce no noticeable degradation in sound quality. It follows from this that most CD players and DAT players sound the same.”

That is all.

bob

[Audio_Empire]

The SR reviews are suspect due to SR's editorial policy which was
that everything printed in SR must serve the advertisers/potential
advertisers. That meant no critical evaluations of anything. Ever
wonder why SR never published a negative review from Julian
Hirsch? Because it was SR policy to not publish negative reviews.
That didn't mean that Julian never came across a piece of equipment
that didn't meet its public specs. It simply meant that SR didn't run
the review, that's all. You see, it was their editorial policy to cater
to the industry, not the consumer. It is because of this policy that
the late J. Gordon Holt founded Stereophile. His stint at High-Fidelity
Magazine (and I believe that he also worked at SR for a time too)
convinced him that these magazines weren't serving the interest
of the consumer. That's also why that no one should be surprised
that SR's tests on the audibility of components, including CD players,
show no differences in audible performance. It's also where the joke
"quote" from Julian Hirsch goes like this: "of all the amplifiers that I
have ever tested, this was one of them" That "quote" applies to
tuners, CD decks, preamps, receivers, you name it. And no, Julian
never really said that, but if you read the sum-total of his work,
including going back to "Hirsch-Houck" labs before Julian went off
on his own, you will see that he never had an opinion. He just
measured the equipment against its published specs, and if it met
them, it was good for go. If not, that fact was never mentioned (as
far as I know and I subscribed to SR for decades!) and the reviews
were not published. The fact that to SR, everything sounded the same
was sacrosanct. I don't wonder that all of those "DBTs" showed no
difference in CD players.

I won't comment on the Sensible Sound tests because I've only seen
a couple of issues of that magazine and don't know what their
editorial policy was.

As for the early Philips (Magnavox) players sounding "different" in
one of those tests, I agree. It did sound different from the early
Japanese players. It was listenable, the early Sonys, Kyoceras,
and Technics players were not and that's MY opinion.

[nab...@hotmail.com]

On Monday, December 10, 2012 6:17:06 PM UTC-5, Audio_Empire wrote:

> The SR reviews are suspect due to SR's editorial policy which was
> that everything printed in SR must serve the advertisers/potential
> advertisers.

Science doesn't rely on editorial policies. Science relies on proper test methodology. Anyone interested can seek out the articles (try either major urban public libraries or technical academic libraries) and see for themselves how well these tests were carried out.

Subsequent research has pretty much vindicated Hirsch, but that's the subject for another thread.

BTW, the idea that a guy who thought all properly functioning amps sounded alike was serving his advertisers is ridiculous. For service to advertisers, Stereophile (along with TAS) takes the cake.

<snip>

> As for the early Philips (Magnavox) players sounding "different" in
> one of those tests, I agree. It did sound different from the early
> Japanese players. It was listenable, the early Sonys, Kyoceras,
> and Technics players were not and that's MY opinion.

The biggest trouble with high-end audio ever since the term was coined is the mistaken confusion of opinion with fact.

bob

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message
news:ka5qh...@news4.newsguy.com...

> The SR reviews are suspect due to SR's editorial policy which was
> that everything printed in SR must serve the advertisers/potential
> advertisers. That meant no critical evaluations of anything. Ever
> wonder why SR never published a negative review from Julian

> Hirsch? Because it was SR policy to not publish negative reviews.]

Looks like Stereo Review is being stigmatized for doing what other magazines
do without being noticed.

For example, virtually every product ever reveiwed by Stereophile this
millenium shows up on their Recommended Components List (RCL)

I personally agree with editors who seem to take the viewpoint that they
don't have any space for reviews of equipment that is substandard.

[Scott]

On Dec 10, 7:35 pm, nabo...@hotmail.com wrote:
> On Monday, December 10, 2012 6:17:06 PM UTC-5, Audio_Empire wrote:
> > The SR reviews are suspect due to SR's editorial policy which was
> > that everything printed in SR must serve the advertisers/potential
> > advertisers.
>
> Science doesn't rely on editorial policies.

That is true but Stereo Review did.

> Science relies on proper test methodology.

That is true but Stereo Review did not.

>Anyone interested can seek out the articles (try either major urban public libraries or technical academic libraries) and see for themselves how well these tests were carried out.

Very poorly. Clearly Stereo review was a publication that had a very
clear preconception about how certain components sound. Clearly Stereo
Review was not a scientific journal and had no proper peer review
process.

Since you are waving the science flag please show us the peer reviewed
published research that has "pretty much vindicated Hirsch."

>
> BTW, the idea that a guy who thought all properly functioning amps sounded alike was serving his advertisers is ridiculous. For service to advertisers, Stereophile (along with TAS) takes the >cake.

Sorry but that is nonsense. Unlike Stereo Review. TAS and Stereophile
were actually willing to print negative reviews of products. early on
neither publication even accepted advertising. So how were they in
"service to advertisers" then?

>
> <snip>
>
> > As for the early Philips (Magnavox) players sounding "different" in
> > one of those tests, I agree. It did sound different from the early
> > Japanese players. It was listenable, the early Sonys, Kyoceras,
> > and Technics players were not and that's MY opinion.
>
> The biggest trouble with high-end audio ever since the term was coined is the mistaken confusion of opinion with fact.
>

Then show us the science that establishes the facts. Until then I will
say back at you. Looks to me like you are mistaking your opinions as
facts.

[Arny Krueger]

"Scott" <S888...@aol.com> wrote in message
news:airi56...@mid.individual.net...

On Dec 10, 7:35 pm, nabo...@hotmail.com wrote:

>Very poorly. Clearly Stereo review was a publication that had a very
> clear preconception about how certain components sound.

That is not clear to me at all.

I am of the opinion that many people are biased against Stereo Review and
make posts like the one above regardless of whatever facts can be brought to
the discussion.

> Clearly Stereo Review was not a scientific journal and had no proper peer
> review process.

Neither are any of the journals you priase such as Stereophile or TAS. The
above statement is obviously an attempt to single out one magazine of many
for a situation that affected them all. In short it supports my supposition
that its author is highly biased against SR.

[nab...@hotmail.com]

On Wednesday, December 12, 2012 9:20:22 AM UTC-5, Scott wrote:

> Very poorly. Clearly Stereo review was a publication that had a very
> clear preconception about how certain components sound. Clearly Stereo
> Review was not a scientific journal and had no proper peer review
> process.

True, but lack of peer review only means that their methodology has not been independently validated; it does not mean that their methodology is flawed. The open-minded audiophile (obviously a minority taste, alas) should read those articles--and all the reports I cited--and decide for himself whether the methodology seems sound.

As for preconceptions, every scientist has some preconception of how his experiment will turn out. If SR's preconception was that all CD players sound alike, they must have been quite surprised to find an exception in their 18986 test!

> Since you are waving the science flag please show us the peer reviewed
> published research that has "pretty much vindicated Hirsch."

Gladly, but, as I said, it is the subject of another thread. Give me a day or two.

> Then show us the science that establishes the facts. Until then I will
> say back at you. Looks to me like you are mistaking your opinions as
> facts.

I did. I presented multiple tests of dozens of devices over a period of decades by three different organizations. It is a fact that none of these tests could show audible differences between conventionally designed CD players and DACs. It is further a fact that no one has ever presented even a single empirically plausible counterexample to these findings. And it is further a fact that a peer-reviewed textbook (and there is nothing more carefully peer-reviewed than a science textbook) agrees with these findings.

bob

[Scott]

On Dec 12, 2:50 pm, nabo...@hotmail.com wrote:
> On Wednesday, December 12, 2012 9:20:22 AM UTC-5, Scott wrote:
> > Very poorly. Clearly Stereo review was a publication that had a very
> > clear preconception about how certain components sound. Clearly Stereo
> > Review was not a scientific journal and had no proper peer review
> > process.
>
> True, but lack of peer review only means that their methodology has not been independently validated; it does not mean that their methodology is flawed.

But we really don't know. Actually we do know. It was quite flawed It
would have never made it through the peer review process. No big deal
but it ain't science.

> The open-minded audiophile (obviously a minority taste, alas) should read those articles--and all the reports I cited--and decide for himself whether the methodology seems sound.

I did back in the day and found them very flawed.

>
> As for preconceptions, every scientist has some preconception of how his experiment will turn out. If SR's preconception was that all CD players sound alike, they must have been quite
> surprised to find an exception in their 18986 test!

I'm sure if they did find an exception they were surprised.

>
> > Since you are waving the science flag please show us the peer reviewed
> > published research that has "pretty much vindicated Hirsch."
>
> Gladly, but, as I said, it is the subject of another thread. Give me a day or two.

I don't see why it won't fit just fine in this thread. But we'll see
what you come up with in a day or two.

>
> > Then show us the science that establishes the facts. Until then I will
> > say back at you. Looks to me like you are mistaking your opinions as
> > facts.
>
> I did.

No, you showed us absolutely no legitimate science. You showed us
nothing more than non scientific articles from non scientific consumer
magazines.

> I presented multiple tests of dozens of devices over a period of decades by three different organizations. It is a fact that none of these tests could show audible differences between
> conventionally designed CD players and DACs. It is further a fact that no one has ever presented even a single empirically plausible counterexample to these findings. And it is further a fact > that a peer-reviewed textbook (and there is nothing more carefully peer-reviewed than a science textbook) agrees with these findings.
>

You cherry picked from anecdotal evidence that has never met the basic
criteria of real scientific research. Pretty far from real science. If
you are going to wave the flag you better bring the goods. You ain't
gonna find the goods in consumer magazines.

[nab...@hotmail.com]

On Thursday, December 13, 2012 6:48:37 AM UTC-5, Scott wrote:

> But we really don't know. Actually we do know. It was quite flawed It
> would have never made it through the peer review process. No big deal
> but it ain't science.

Flawed in what specific ways? And, no, "the researchers had a pre-test hypothesis about the outcome" is not a flaw. If it were, there would be no science.

Just as a point of comparison, what did the SR folks do wrong in each of their *three* tests that these guys did right:

http://www.aes.org/e-lib/browse.cfm?elib=14195

Needless to say, the latter *did* pass peer review.

<snip>

> You cherry picked

I did? What evidence did I ignore?

> rom anecdotal evidence that has never met the basic
> criteria of real scientific research. Pretty far from real science. If
> you are going to wave the flag you better bring the goods. You ain't
> gonna find the goods in consumer magazines.

One would think a widely used college science textbook would count as "real science." And one would think that if consumer magazines are getting the same results reported by "real scientists," the magazines must be doing something right.

Once again, where is the counterevidence? Where are the DBTs that show these devices to be audibly distinguishable? I won't even hold you to meeting the strictest requirements of peer review. Just show me something empirically plausible.

bob

[Audio_Empire]

On Monday, December 10, 2012 7:35:04 PM UTC-8, nab...@hotmail.com wrote:
> On Monday, December 10, 2012 6:17:06 PM UTC-5, Audio_Empire wrote:
>
>
>
> > The SR reviews are suspect due to SR's editorial policy which was
>
> > that everything printed in SR must serve the advertisers/potential
>
> > advertisers.
>
>
>
> Science doesn't rely on editorial policies.

No, but publications do.

Science relies on proper test methodology. Anyone interested can seek out the articles (try either major urban public libraries or technical academic libraries) and see for themselves how well these tests were carried out.

The idea that a suite of tests which only seeks to confirm a set of
published specs for a unit under test is not, in my estimation, good
science. The further fact that The editorial policy at both SR and
High-Fidelity was that if a component didn't meet specs, the review
was not published, is also not doing good service to one's readership.

Really? Science has vindicated a non-critical approach to evaluation? Since when?

>
>
>
> BTW, the idea that a guy who thought all properly functioning amps sounded alike was serving his advertisers is ridiculous. For service to advertisers, Stereophile (along with TAS) takes the cake

Well there you are wrong. I have written for both TAS and Stereophile
over the years, and no one ever told me how to slant a review. If I
found something negative, I said so in no uncertain terms and they
both published those reviews with all my comments intact. Both
Stereophile and TAS started out accepting NO ads, then they
"graduated" to taking ads only from dealers, and finally from
manufacturers. Both magazines' policy toward advertisers is pretty
much the same: We'll take your ads with the understanding that the
fact that you are an advertiser will have no bearing on the outcome
of reviews of your products. Both magazines have a list of not a few
manufacturers who refuse to advertise with them and won't send
them equipment to review any more because they previously
received a bad review at the hands of one or the other.

.
>
>
>
> <snip>
>
>
>
> > As for the early Philips (Magnavox) players sounding "different" in
>
> > one of those tests, I agree. It did sound different from the early
>
> > Japanese players. It was listenable, the early Sonys, Kyoceras,
>
> > and Technics players were not and that's MY opinion.
>
>
>
> The biggest trouble with high-end audio ever since the term was coined is the mistaken confusion of opinion with fact.

I would say that's more than somewhat true. But often,
opinions merely mirror facts. Cable elevators, green
marking pens, blocks of myrtle wood placed on the tops
of components, "cryogenically treated" clocks
and cable sound are all unsupported mythology, but early CD
players that sounded nasty to a rather large group
of people definitely mirror facts. Let's face it, not everyone is
a critical listener. That's a facility that one has to nurture, its
not God-given. And as was discussed ad-nauseum in another
thread, there are people who are so biased toward certain
precepts that they wouldn't hear things that challenged their
biases even if that characteristic stuck-out like a sore thumb!

[Audio_Empire]

On Tuesday, December 11, 2012 2:22:08 PM UTC-8, Arny Krueger wrote:
> "Audio_Empire" <gmgr...@comcast.net> wrote in message
>
> news:ka5qh...@news4.newsguy.com...
>
>
>
> > The SR reviews are suspect due to SR's editorial policy which was
>
> > that everything printed in SR must serve the advertisers/potential
>
> > advertisers. That meant no critical evaluations of anything. Ever
>
> > wonder why SR never published a negative review from Julian
>
> > Hirsch? Because it was SR policy to not publish negative reviews.]
>
>
>
> Looks like Stereo Review is being stigmatized for doing what other magazines
>
> do without being noticed.

What magazines would they be, Mr. Kruger?

>
>
>
> For example, virtually every product ever reveiwed by Stereophile this
>
> millenium shows up on their Recommended Components List (RCL)

That's simply a very misleading statement. (1) Not everything published
in Stereophile makes it to the Recommended Components list. and (2)
those that do are categorized according their perceived flaws and listed
under an alphabetical hierarchy. To wit: "A" is state of the art, and "D" is
very flawed but still acceptable. I've seen lots of critical reviews in
Stereophile.

>
>
>
> I personally agree with editors who seem to take the viewpoint that they
>
> don't have any space for reviews of equipment that is substandard.

And that serves the readership, how? Seems to me that serves the
advertisers. "Yeah, your new amplifier is lousy, but we won't tell
anybody about it. OK? And while were on the phone, you want to
buy a new ad?"

Gimme a break!

[Audio_Empire]

I don't remember anybody praising either TAS or Stereophile. All magazines
fall under the heading of "Entertainment", and should be taken with a grain
of salt. The only thing that Stereophile and TAS does that's different from
Stereo Review and High-Fidelity is that if a review of a piece of equipment
comes out negative, they publish it. OTOH, both SR and High-Fidelity were
better reads than either Stereophile or TAS. I learned a lot about music and
musicians from those "slicks". HF especially, was once a very classy publication.
They had writers like Gene Lees writing articles about jazz, and latin music, and
writers like Nicholas Slonimsky writing about classical music and classical
artists.

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kadlv...@news3.newsguy.com...

> I don't remember anybody praising either TAS or Stereophile.

In another post you mention writing for one or both of these publications.

If you cannot bring yourself to praise them, how can you bring yourself to
write for them?

Or, was the money they paid sufficient to induce you to write for a
publication that was so bad that you did not think they were praiseworthy?

[Sebastian Kaliszewski]

On 12/13/2012 12:48 PM, Scott wrote:
> On Dec 12, 2:50 pm, nabo...@hotmail.com wrote:
>> On Wednesday, December 12, 2012 9:20:22 AM UTC-5, Scott wrote:
>>> Very poorly. Clearly Stereo review was a publication that had a
>>> very clear preconception about how certain components sound.
>>> Clearly Stereo Review was not a scientific journal and had no
>>> proper peer review process.
>>
>> True, but lack of peer review only means that their methodology
>> has not been independently validated; it does not mean that their
>> methodology is flawed.
>
>
> But we really don't know. Actually we do know. It was quite flawed
> It would have never made it through the peer review process. No big
> deal but it ain't science.
>
>
>> The open-minded audiophile (obviously a minority taste, alas)
>> should read those articles--and all the reports I cited--and decide >> for
himself whether the methodology seems sound.
>
>
> I did back in the day and found them very flawed.

Would you care to present those alleged flaws?

>
>
>>
>> As for preconceptions, every scientist has some preconception of
>> how his experiment will turn out. If SR's preconception was that all >> CD
players sound alike, they must have been quite
>> surprised to find an exception in their 18986 test!
>
> I'm sure if they did find an exception they were surprised.

They did and published that. Contrary to what you and Mr. Audio Empire
stated more than once about them that they never would.

>>
>>> Since you are waving the science flag please show us the peer
>>> reviewed published research that has "pretty much vindicated
>>> Hirsch."
>>
>> Gladly, but, as I said, it is the subject of another thread. Give
>> me a day or two.
>
> I don't see why it won't fit just fine in this thread. But we'll see
> what you come up with in a day or two.

So please include in this very thread scientific (or even half way
ameteurish-sceintific) evidence to support your stance.

>
>
>>
>>> Then show us the science that establishes the facts. Until then I
>>> will say back at you. Looks to me like you are mistaking your
>>> opinions as facts.
>>
>> I did.
>
> No, you showed us absolutely no legitimate science. You showed us
> nothing more than non scientific articles from non scientific
> consumer magazines.

You apparently missed this one:

Moore, BCJ. An Introduction to the Psychology of Hearing, Fourth
Edition. San Diego: Academic Press, 1997.

>
>

>> I presented multiple tests of dozens of devices over a period of
>> decades by three different organizations. It is a fact that none of
>> these tests could show audible differences between
>> conventionally designed CD players and DACs. It is further a fact
>> that no one has ever presented even a single empirically plausible
>> counterexample to these findings. And it is further a fact that a
>> peer-reviewed textbook (and there is nothing more carefully
>> peer-reviewed than a science textbook) agrees with these findings.
>>
> You cherry picked from anecdotal evidence that has never met the
> basic criteria of real scientific research. Pretty far from real
> science. If you are going to wave the flag you better bring the
> goods. You ain't gonna find the goods in consumer magazines.
>

See above.


Then...

Lack of any scientific (or even halfway amateur-sceintific like the one
you're criticizing) evidence to support your position noted.

rgds
\SK

[Scott]

On Dec 13, 2:44 pm, nabo...@hotmail.com wrote:
> On Thursday, December 13, 2012 6:48:37 AM UTC-5, Scott wrote:
> > But we really don't know. Actually we do know. It was quite flawed It
> > would have never made it through the peer review process. No big deal
> > but it ain't science.
>
> Flawed in what specific ways? And, no, "the researchers had a pre-test hypothesis about the outcome" is not a flaw. If it were, there would be no science.

It has been quite a while since that article came out. I would be
happy to review it again and point out the problems I found with it at
the time if you would care to email me a copy. I don't make a point of
memorizing these things for decades to come.

>
> Just as a point of comparison, what did the SR folks do wrong in each of their *three* tests that these guys did right:
>
> http://www.aes.org/e-lib/browse.cfm?elib=14195

I would be happy to read that paper and compare it to the Stereo
Review article if you'd like to email me a copy of that one as well.

>
> Needless to say, the latter *did* pass peer review.
>

Which certainly does give it far more credibility on a scientific
level.

>
> > You cherry picked
>
> I did? What evidence did I ignore?

Do you really think you got them all?

>
> > rom anecdotal evidence that has never met the basic
> > criteria of real scientific research. Pretty far from real science. If
> > you are going to wave the flag you better bring the goods. You ain't
> > gonna find the goods in consumer magazines.
>
> One would think a widely used college science textbook would count as "real science."

It would likely count as a book that talks about real science. What on
earth does that have to do with the articles you cited from consumer
magazines?

> And one would think that if consumer magazines are getting the same results reported by "real scientists," the magazines must be doing something right.
>

1. Not sure why one would think that. One could just as easily get the
same results with a coin flip. 2. I'm not so sure they were testing
the same things nor getting "the same results." 3. One peer reviewed
paper on one set of listening tests is certainly legitimate scientific
evidence. It is not something one uses to declare a final dogmatic
objective fact.

> Once again, where is the counterevidence? Where are the DBTs that show these devices to be audibly distinguishable? I won't even hold you to meeting the strictest requirements of peer > > review. Just show me something empirically plausible.

Why would you not hold me to the strictest requirements? Are we going
to stick with science or swap anecdotes?
I would be happy to read the one peer reviewed paper you have now
brought up. But the articles from Stereo Review are junk in the world
of real science. Doesn't matter if they wrought similar results to one
legitimate set of tests. And if you want to send me copies of the
articles by Stereo review I would be happy to review them point out
the problems I saw with them back in the day.

[Scott]

On Dec 14, 3:55 am, Sebastian Kaliszewski

<Sebastian.Kaliszew...@softax.pl> wrote:
> On 12/13/2012 12:48 PM, Scott wrote:> On Dec 12, 2:50 pm, nabo...@hotmail.com wrote:
> >> On Wednesday, December 12, 2012 9:20:22 AM UTC-5, Scott wrote:
> >>> Very poorly. Clearly Stereo review was a publication that had a
> >>> very clear preconception about how certain components sound.
> >>> Clearly Stereo Review was not a scientific journal and had no
> >>> proper peer review process.
>
> >> True, but lack of peer review only means that their methodology
> >> has  not been independently validated; it does not mean that their
>
>  >> methodology is flawed.
>
> > But we really don't know. Actually we do know. It was quite flawed
> > It would have never made it through the peer review process. No big
> > deal but it ain't science.
>
> >> The open-minded audiophile (obviously a minority taste, alas)
> >> should  read those articles--and all the reports I cited--and decide >> for
>
> himself whether the methodology seems sound.
>
>
>
> > I did back in the day and found them very flawed.
>
> Would you care to present those alleged flaws?

I would be happy to if someone wants to send me a copy of the old
article to jog my memory. If it aint Shakespeare or something like it
I rarely memorize such things.

>
>
>
> >> As for preconceptions, every scientist has some preconception of
> >> how  his experiment will turn out. If SR's preconception was that all >> CD
>
> players sound alike, they must have been quite
>
> >> surprised to find an exception in their 18986 test!
>
> > I'm sure if they did find an exception they were surprised.
>
> They did and published that. Contrary to what you and Mr. Audio Empire
> stated more than once about them that they never would.
>
>
>
> >>> Since you are waving the science flag please show us the peer
> >>> reviewed published research that has "pretty much vindicated
> >>> Hirsch."
>
> >> Gladly, but, as I said, it is the subject of another thread. Give
> >> me  a day or two.
>
> > I don't see why it won't fit just fine in this thread. But we'll see
> > what you come up with in a day or two.
>
> So please include in this very thread scientific (or even half way
> ameteurish-sceintific) evidence to support your stance.

Do you actually understand my stance? If so I think you would have no
trouble finding a mountain of support from the scientific community in
support of "my stance."

>
>
>
> >>> Then show us the science that establishes the facts. Until then I
> >>> will say back at you. Looks to me like you are mistaking your
> >>> opinions as facts.
>
> >> I did.
>
> > No, you showed us absolutely no legitimate science. You showed us
> > nothing more than non scientific articles from non scientific
> > consumer magazines.
>
> You apparently missed this one:
> Moore, BCJ. An Introduction to the Psychology of Hearing, Fourth
> Edition. San Diego: Academic Press, 1997.

That wasn't on Bob's list in his OP.

>
>
>
> >> I presented multiple tests of dozens of devices over a period of
>
>  >> decades by three different organizations. It is a fact that none of
>  >> these tests could show audible differences between>> conventionally designed CD players and DACs. It is further a fact
>
>  >> that no one has ever presented even a single empirically plausible
>  >> counterexample to these findings. And it is further a fact that a
>  >> peer-reviewed textbook (and there is nothing more carefully
>  >> peer-reviewed than a science textbook) agrees with these findings.
>
>
>
> > You cherry picked from anecdotal evidence that has never met the
> > basic criteria of real scientific research. Pretty far from real
> > science. If you are going to wave the flag you better bring the
> > goods. You ain't gonna find the goods in consumer magazines.
>
> See above.

See my answer to the above, above.

>
> Then...
>
> Lack of any scientific (or even halfway amateur-sceintific like the one
> you're criticizing) evidence to support your position noted.

Really? Me thinks you don't really understand "my position" if you
think it is not supported by science. Just for kicks lets see if you
can review the thread and accurately restate my position. We can go
from there.

[Doug McDonald]

One of the problems I see in many ABX double blind tests,
especially but not exclusively speakers, is level matching.

In some cases careful level matching may be a proper thing to do.
But in others I contend it is not.

The problem with trying to match levels is that level differences
are usually easy to spot. And this applies to frequency response
differences.

I contend that a better way, though much harder, to reliably
detect small differences of all kinds is to set the overall
level of A and B to be the same, and then for each trial of
A, B, or X to vary the level by a random amount of 0 +- 1 or
0 +- 2 dB. Of course this requires lots of trials to get
good statistics.

But once done, if the test is positive and the participants
decide that A has let's say "better imaging" then it is much clearer
that what they are not doing is deciding, for example, that
louder has better imaging, because they would have to hear the imaging
effect "through" differences in level.

Etc.

Doug McDonald

[Jenn]

In article <kadlv...@news3.newsguy.com>,

Audio_Empire <gmgr...@comcast.net> wrote:

>Nicholas Slonimsky writing about classical music and classical
> artists.

Slonimsky was an absolutely one of a kind wonder of the world. He was
also my composition teacher in the 70s.

[Scott]

I don't think level matching is ever the wrong thing to do in an *ABX*
DBT. It is important to understand that the only purpose an *ABX* DBT
serves is to test for audible differences not for preferences. With
blind *preference* tests level matching is a complicated issue as you
point out.

[Audio_Empire]

On Thursday, December 13, 2012 7:14:05 PM UTC-8, Arny Krueger wrote:
> "Audio_Empire" <gmgr...@comcast.net> wrote in message
>
> news:kadlv...@news3.newsguy.com...
>
>
>
> > I don't remember anybody praising either TAS or Stereophile.
>
>
>
> In another post you mention writing for one or both of these publications.

Yeah? so?

>
>
>
> If you cannot bring yourself to praise them, how can you bring yourself to
>
> write for them?

Who said that I can't bring myself to praise them? All I said was that nobody
in these recent threads had praised them. OTOH, I have defended them
against contributors who make statements about their editorial policies
that aren't correct, but that's not praise, it's merely setting the record
straight. Someone else made the same points I did, and they weren't praising
them either.

>
>
>
> Or, was the money they paid sufficient to induce you to write for a
> publication that was so bad that you did not think they were praiseworthy?

My reasons for leaving them, as well as my monetary renumeration are
no one's business but mine, but I will say that it was due to none of the
above.

[Audio_Empire]

I think it's de riguer to match levels in any type of formal listening tests and it
needs to be done within a half of a dB – or better. The human ear will always
pick out the louder component as being the better of the two, and of course,
with a level difference, in an ABX test the listeners will always say there's a
difference between the devices under test, even if they are both two samples
of the same make and model!

[Scott]

for the sake of ABX for sure. OTOH for preference comparisons we
simply may have real problems doing a true level match. Let's say we
are comparing two different masterings of the same title. One is
compressed and both have substantially different EQ. How do you level
match? Peak levels? average levels? At what frequency?

[Barkingspyder]

[ A large number of blank lines were trimmed out of this
response. -- dsr ]

Where does this knowledge come from?

That meant no critical evaluations of anything. Ever
>
> wonder why SR never published a negative review from Julian
>
> Hirsch? Because it was SR policy to not publish negative reviews.
>

Same question.

> That didn't mean that Julian never came across a piece of equipment
>
> that didn't meet its public specs. It simply meant that SR didn't run
>
> the review, that's all. You see, it was their editorial policy to cater
>
> to the industry, not the consumer.

So they were just like Stereophile?

It is because of this policy that
>
> the late J. Gordon Holt founded Stereophile. His stint at High-Fidelity
>
> Magazine (and I believe that he also worked at SR for a time too)
>
> convinced him that these magazines weren't serving the interest
>
> of the consumer. That's also why that no one should be surprised
>
> that SR's tests on the audibility of components, including CD players,
>
> show no differences in audible performance.

Or maybe it's because there are so few instances to report.

It's also where the joke
>
> "quote" from Julian Hirsch goes like this: "of all the amplifiers that I
>
> have ever tested, this was one of them" That "quote" applies to
>
> tuners, CD decks, preamps, receivers, you name it. And no, Julian
>
> never really said that, but if you read the sum-total of his work,
>
> including going back to "Hirsch-Houck" labs before Julian went off
>
> on his own, you will see that he never had an opinion. He just
>
> measured the equipment against its published specs, and if it met
>
> them, it was good for go. If not, that fact was never mentioned (as
>
> far as I know and I subscribed to SR for decades!) and the reviews
>
> were not published. The fact that to SR, everything sounded the same
>
> was sacrosanct. I don't wonder that all of those "DBTs" showed no
>
> difference in CD players.
>

I distinctly recall a message in SR from Mr. Hirsch commenting on the fact that there virtually no negative reviews. It was because virtually everything does sound the same. I also recall that there were reviews that criticized various things just can't recall what.

>
>
> I won't comment on the Sensible Sound tests because I've only seen
>
> a couple of issues of that magazine and don't know what their
>
> editorial policy was.
>
>
>
> As for the early Philips (Magnavox) players sounding "different" in
>
> one of those tests, I agree. It did sound different from the early
>
> Japanese players. It was listenable, the early Sonys, Kyoceras,
>
> and Technics players were not and that's MY opinion.

The facts as I have discovered are this, that when components perform within proper parameters nobody can hear a difference reliably. When they operate outside of those parameters they can be equalized so that they do and then the differences are no longer detectable. You are entitled to you opinion of course, just recognize that it is at odds with what is known. If it sounds different it's either because it is not designed to perform the way it should or it's broken.

[Barkingspyder]

[ A large number of blank lines were trimmed out of this

article. Please do suitable editing in future. -- dsr ]

On Friday, December 14, 2012 1:38:20 PM UTC-8, Scott wrote:
> On Dec 14, 12:29 pm, Audio_Empire <gmgrav...@comcast.net> wrote:
>
> > On Friday, December 14, 2012 11:11:42 AM UTC-8, Scott wrote:
>
> > > On Dec 14, 9:07 am, Doug McDonald <mcdon...@scs.uiuc.edu> wrote:
> >
> > > > One of the problems I see in many ABX double blind tests,
> > > > especially but not exclusively speakers, is level matching.
> > > > In some cases careful level matching may be a proper thing to do.
> > > > But in others I contend it is not.
> > > > The problem with trying to match levels is that level differences
> > > > are usually easy to spot. And this applies to frequency response
> > > > differences.
> >
> > > > I contend that a better way, though much harder, to reliably
> > > > detect small differences of all kinds is to set the overall
> > > > level of A and B to be the same, and then for each trial of
> > > > A, B, or X to vary the level by a random amount of 0 +- 1 or
> > > > 0 +- 2 dB.  Of course this requires lots of trials to get
> > > > good statistics.
>
> >
>
> > > > But once done, if the test is positive and the participants

> > > > decide that A has let's say "better imaging" then it is much cleare> > > > that what they are not doing is deciding, for example, that

> > > > louder has better imaging, because they would have to hear the imaging
> > > > effect "through" differences in level.
> > > > Etc.
> > > > Doug McDonald
>
> >
>
> > > I don't think level matching is ever the wrong thing to do in an *ABX*
> > > DBT. It is important to understand that the only purpose an *ABX* DBT
> > > serves is to test for audible differences not for preferences. With
> > > blind *preference* tests level matching is a complicated issue as you
> > > point out.
>
> >
>
> > I think it's de riguer to match levels in any type of formal listening tests and it
> > needs to be done within a half of a dB – or better. The human ear will always
> > pick out the louder component as being the better of the two, and of course,
> > with a level difference, in an ABX test the listeners will always say there's a
> > difference between the devices under test, even if they are both two samples
> > of the same make and model!
>
>
> for the sake of ABX for sure. OTOH for preference comparisons we
> simply may have real problems doing a true level match. Let's say we
> are comparing two different masterings of the same title. One is
> compressed and both have substantially different EQ. How do you level
> match? Peak levels? average levels? At what frequency?

The nice thing about testing for difference as ABX does is that if there is no difference detected you know that the more expensive one is not any better sounding. Unless it has features you feel you must have or you just like the look better you can save some money. Personally, I like knowing that a $2000.00 set of electronics is not going to be out performed by a $20,000.00 set. Speakers of course, (the part that you actually hear in a sound system) are another story entirely.

[Audio_Empire]

Well obviously I was talking about equipment evaluations, not source
evaluations. Problem here is that there is no real standard for recordings
They seem to be all over the map. So yes, that would be difficult. Even if
you used a test tape (for tape decks) or a test CD to calibrate the CD
players, it doesn't mean anything unless the recordings in question were
calibrated to the same standard. With tape, this was possible in the days
of Dolby "A" or Dolby "B" because the tapes had a Dolby calibration tone
at the beginning and the end of the tape. Sadly commercial CDs don't
have that.

[Barkingspyder]

On Thursday, December 13, 2012 2:47:34 PM UTC-8, Audio_Empire wrote:
> On Tuesday, December 11, 2012 2:22:08 PM UTC-8, Arny Krueger wrote:
>
> > "Audio_Empire" <gmgr...@comcast.net> wrote in message
>
> >
>
> > news:ka5qh...@news4.newsguy.com...
>
> >
>
> >
>
> >
>
> > > The SR reviews are suspect due to SR's editorial policy which was
>
> >
>
> > > that everything printed in SR must serve the advertisers/potential
>
> >
>
> > > advertisers. That meant no critical evaluations of anything. Ever
>
> >
>
> > > wonder why SR never published a negative review from Julian
>
> >
>
> > > Hirsch? Because it was SR policy to not publish negative reviews.]
>
> >
>
> >
>
> >
>

> > I personally agree with editors who seem to take the viewpoint that they
>
> >
>
> > don't have any space for reviews of equipment that is substandard.
>
>
>
> And that serves the readership, how? Seems to me that serves the
>
> advertisers. "Yeah, your new amplifier is lousy, but we won't tell
>
> anybody about it. OK? And while were on the phone, you want to
>
> buy a new ad?"
>
>
>
> Gimme a break!

It serves them in that they know if it has been reviewed it can be trusted to perform as it supposed to, no audible coloration other than for tubed gear turntables and phono cartridges and tape decks. I still remember the first CD I ever heard and I knew I had to have one if for no other reason than the absence of surface noise. There was so much more though, the clearer sound of everything, the attack of the percussion, and especially, the bass.

[Scott]

On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:

>
> The nice thing about testing for difference as ABX does is that if there is no difference detected you know that the more expensive one is not any better sounding.  Unless it has features you > feel you must have or you just like the look better you can save some money.  Personally, I like knowing that a $2000.00 set of electronics is not going to be out performed by a $20,000.00 set. > Speakers of course, (the part that you actually hear in a sound system) are another story entirely.

heck if it makes you feel better about buying less expensive gear I
guess that's nice. But you are putting way too much weight on such a
test if you think you walk away from a single null result "knowing"
that the more expensive gear is not better sounding. But hey, if it
makes you happy that's great. But not everyone is on board with you
there.

[Scott]

The person who was questioning to value of level matching did not seem
to be limiting his opinion to CDPs and amps. You still have the same
problems in level matching that I stated above when dealing with
loudspeakers. In fact you have even more problems with radiation
pattern differences and room interfaces that make it even more
impossible to do a true level match.

[Audio_Empire]

On Friday, December 14, 2012 8:14:31 PM UTC-8, Barkingspyder wrote:
> [ A large number of blank lines were trimmed out of this
>
> response. -- dsr ]
>
>
>
>
>
> On Monday, December 10, 2012 3:17:06 PM UTC-8, Audio_Empire wrote:
>
> > On Sunday, December 9, 2012 7:59:31 PM UTC-8, nab...@hotmail.com wrote:

> > The SR reviews are suspect due to SR's editorial policy which was
>
> >
>
> > that everything printed in SR must serve the advertisers/potential
>
> >
>
> > advertisers.
>
>
>
> Where does this knowledge come from?

J. Gordon Holt who worked for High-Fidelity and had friends who worked
for SR told me this many years ago. But even so, if you had read these rags, i
t would be very apparent to even the most casual observer.

>
>
>
>
>
> That meant no critical evaluations of anything. Ever

Pretty much the way it has to be if you are merely a pipeline
for the industry's public relations.

> > wonder why SR never published a negative review from Julian

> > Hirsch? Because it was SR policy to not publish negative reviews.

> Same question.

Well, they never did publish a negative revue. If a piece of equipment
didn't meet it's published specs on the bench, the review never made it to
print. That's just the way it was.

>
>
>
> > That didn't mean that Julian never came across a piece of equipment
> > that didn't meet its public specs. It simply meant that SR didn't run
> > the review, that's all. You see, it was their editorial policy to cater
> > to the industry, not the consumer.
>

> So they were just like Stereophile?

No. Stereophile seems to grade components these days. If there are flaws,
the flaws are mentioned in the review. Neither SR or HF ever did that.

>
>> It is because of this policy that
>> the late J. Gordon Holt founded Stereophile. His stint at High-Fidelity
> > Magazine (and I believe that he also worked at SR for a time too)
> > convinced him that these magazines weren't serving the interest
> > of the consumer. That's also why that no one should be surprised
> > that SR's tests on the audibility of components, including CD players,
> > show no differences in audible performance.

> Or maybe it's because there are so few instances to report.

Not likely. These magazines have been gone for a long time
During their heyday, there were lots of lousy components. Take
for instance the Dynaco Stereo 120 power amp. The original one
was lousy sounding (it had a crossover notch, fer chrissake!) and
unreliable. But Julian Hirsch said it was great.

> > It's also where the joke
> > "quote" from Julian Hirsch goes like this: "of all the amplifiers that I
> > have ever tested, this was one of them" That "quote" applies to
> > tuners, CD decks, preamps, receivers, you name it. And no, Julian
> > never really said that, but if you read the sum-total of his work,
> > including going back to "Hirsch-Houck" labs before Julian went off
> > on his own, you will see that he never had an opinion. He just
> > measured the equipment against its published specs, and if it met
> > them, it was good for go. If not, that fact was never mentioned (as
> > far as I know and I subscribed to SR for decades!) and the reviews
> > were not published. The fact that to SR, everything sounded the same
> > was sacrosanct. I don't wonder that all of those "DBTs" showed no
> > difference in CD players.

> I distinctly recall a message in SR from Mr. Hirsch commenting on the fact that there virtually no negative reviews. It was because virtually everything does sound the same. I also recall that there were reviews that criticized various things just can't recall what.

That's balderdash, especially in SR's hey-day but still, it was SR's editorial stance: "Everything
sounds the same." That might have some truth to it today
(differences still exist, but they are very subtle and as I have said
before, largely of little consequence).

> > I won't comment on the Sensible Sound tests because I've only seen
> > a couple of issues of that magazine and don't know what their
> > editorial policy was.
>
> > As for the early Philips (Magnavox) players sounding "different" in
> > one of those tests, I agree. It did sound different from the early
> > Japanese players. It was listenable, the early Sonys, Kyoceras,
> > and Technics players were not and that's MY opinion.

> The facts as I have discovered are this, that when components perform within proper parameters nobody can hear a difference reliably. When they operate outside of those parameters they can be equalized so that they do and then the differences are no longer detectable. You are entitled to you opinion of course, just recognize that it is at odds with what is known. If it sounds different it's either because it is not designed to perform the way it should or it's broken.

Well nobody can help those people who's biases cause them to leave their critical
facilities at the door and either cannot or will not hear what is there for all to hear.

[Audio_Empire]

On Friday, December 14, 2012 8:22:43 PM UTC-8, Barkingspyder wrote:
> On Thursday, December 13, 2012 2:47:34 PM UTC-8, Audio_Empire wrote:
>
> > On Tuesday, December 11, 2012 2:22:08 PM UTC-8, Arny Krueger wrote:

>
> > And that serves the readership, how? Seems to me that serves the
> > advertisers. "Yeah, your new amplifier is lousy, but we won't tell
> > anybody about it. OK? And while were on the phone, you want to
> > buy a new ad?"
>
> > Gimme a break!
>
>
>
> It serves them in that they know if it has been reviewed it can be trusted to perform as it supposed to, no audible coloration other than for tubed gear turntables and phono cartridges and tape decks.

Oh if that were only true!

> I still remember the first CD I ever heard and I knew I had to have one if for no other reason than the absence of surface noise. There was so much more though, the clearer sound of everything, the attack of the percussion, and especially, the bass.

I still remember the first CD player I ever heard it was the Winter CES
in 1981 or 1982. It was a Sony prototype and my reaction was Yecch!
Today of course, most players are quite good, and it is possible to
master CDs that are so good that if they had done CDs that way
across the entire industry, there would have been no need to develop
SACD or DVD-A. Try one of the JVC XRCDs. They are truly state-of-
the-art. And the only thing special about them is that they were
very carefully mastered and manufactured. Most of todays CDs are
terribly compressed and limited and sound lousy. Even some modern
remastering of classic recordings from many pop stars. Contrast the
latest re-mastering of these performances with the earlier CD releases of
the same titles, and you'll see what I mean. CD can sound glorious
if done right. Too bad it so seldom is. Even for classical music.

[Audio_Empire]

My experience is that with speakers, DBTs really aren't necessary. Speakers are all over the place
with respect to frequency response, distortion, radiation pattern, and sensitivity (efficiency), that it is a given that no two models sound the same. Speakers are best evaluated in one's own listening environment and over a period of several days. Not convenient, but because speakers are a system
heard in conjunction with the room in which they are playing, it is, alas necessary (but seldom done).

[Andrew Haley]

Audio_Empire <gmgr...@comcast.net> wrote:
> My experience is that with speakers, DBTs really aren't necessary.

I don't think it's quite that, it's more that it's very hard to do.
Harman famously made a machine that could quickly exchange speakers so
that they could be compared the same position. With an opaque cloth,
this removed the physical appearance of the speakers from the
comparison so that the speakers could be evaluated by sound alone.
Audio reviewers could do the same.

Andrew.

[Audio_Empire]

My sentiments exactly. I'm convinced that while DBTs work great for drug tests, tests by food manufacturers about new or altered products, etc., I'm not terribly sure that they work for audio equipment because the waveform that we are "analyzing" with our collective ears is pretty complex. Now I'm sure that cables and interconnects are the exception. They are supposed to be simple conductors and therefore, going by the physics of conductors and their performance over a frequency
range, which are known quantities, they aren't supposed to have any affect on the signal at audio frequencies, and so, in a DBT they demonstrate that they don't.

Otherwise for DACs, preamps and amps, there are certainly differences (in DACs, especially) yet they don't show-up in DBTs and ABX tests. Granted, with modern, solid-state amps and preamps the differences are minute (and largely inconsequential), but they do show themselves in properly set up
DBT tests. Often it takes more than a few seconds of listening before the DUTs are switched, and some
characteristics like imaging and soundstage might not show-up at all with some types of music or certain recordings, but under the right circumstances these things can be heard in DBT. I've proved that
many times to MY OWN satisfaction.

[Jenn]

In article <kakn7...@news6.newsguy.com>,

So that the speakers could be in the same position? The problem with
this seems obvious.

[Scott]

On Dec 14, 8:14 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:

> The facts as I have discovered are this, that when components perform within proper parameters nobody can hear a difference reliably.  When they operate outside of those parameters they
> can be equalized so that they do and then the differences are no longer detectable.  You are entitled to you opinion of course, just recognize that it is at odds with what is known.  If it
> sounds different it's either because it is not designed to perform the way it should or it's broken.

What you have discovered there are your personal opinions not facts.
And you are welcome to those opinions. But the "fact" is there is
plenty of components that have a distinctive sonic signature even
while operating within their intended limitations. Plenty of tube amps
and analog source components have distinctive sonic signatures that
you can't duplicate with EQ. These components are neither "broken" nor
are they failing to perform as designed. As to how components *should*
be designed to perform is a matter of opinion not a matter of fact as
well. Ultimately the criteria is whether or not the consumer likes
what they hear. We as audiophiles are under no obligation to tailor or
preferences to some arbitrary standards of measured performance.

[Scott]

On Dec 16, 10:10 am, Jenn <jennconductsREMOVET...@mac.com> wrote:
> In article <kakn7o01...@news6.newsguy.com>,
>  Andrew Haley <andre...@littlepinkcloud.invalid> wrote:

>
> > Audio_Empire <gmgrav...@comcast.net> wrote:
> > > My experience is that with speakers, DBTs really aren't necessary.
>
> > I don't think it's quite that, it's more that it's very hard to do.
> > Harman famously made a machine that could quickly exchange speakers so
> > that they could be compared the same position.  With an opaque cloth,
> > this removed the physical appearance of the speakers from the
> > comparison so that the speakers could be evaluated by sound alone.
> > Audio reviewers could do the same.
>
> > Andrew.
>
> So that the speakers could be in the same position?  The problem with
> this seems obvious.

As has been pointed out. There really is no need to do ABX DBTs for
speakers. However the idea of doing blind preference tests for
speakers I think is quite worthwhile. But....it is also incredibly
difficult to do without stacking the deck. This *is* where level
matching becomes quite a complicated issue. And as you are alluding to
so does speaker position. Add to that the room itself which may favor
one speaker system over another and you have a very very difficult
task. And then of course the physical aspect of changing out speakers
quickly without giving away which are which.

Certainly the HK facility which allows for quick switching double
blind comparisons is state of the art. But even that has it's
limitations. Unfortunately I think the methodologies used there are
very problematic as well. But that is another thread.

[Audio_Empire]

Ultimately, what most of us are after, is a system that sounds to us like real
music played in a real space. There are many different versions of that goal
and they are each valid to SOME listener. Some like their sound lush and romantic
and those people are drawn to classic tube designs for their electronics (like
a Chinese Yaqin MC-100B), others like their sound cool and analytical and
they would go for some solid-state design known for that kind of sound (like
Krell) and some would want their sound as neutral as possible (Nelson Pass).
As you say, all of these amps have different sonic signatures, and those
signatures don't necessarily reveal themselves in a DBT test (although some
will). Most of the differences are very subtle and many are, in the final
analysis, trivial. But one can definitely hear the difference between a Yaqin
MC-100b and a Krell i300 because while the Yaqin sounds very lush and
"musical", it is definitely "colored" and the Krell is squeaky clean. Horses
for courses and all that.

[Audio_Empire]

I still maintain that DBTs aren't necessary for speaker evaluation, but I do
maintain that it is very necessary (and eminently desirable) to evaluate
speakers in your OWN listening room. That's what is difficult to do (not
that DBTs on speakers would be easy or convenient) because: (A) few
dealerships will let you borrow large speakers, and (B) even if you did find
an accommodating dealer, floor standers are usually heavy and difficult to
transport. That leaves only "mini monitor" types.

[Andrew Haley]

Jenn <jennconduct...@mac.com> wrote:
> In article <kakn7...@news6.newsguy.com>,
> Andrew Haley <andr...@littlepinkcloud.invalid> wrote:
>
>> Audio_Empire <gmgr...@comcast.net> wrote:
>> > My experience is that with speakers, DBTs really aren't necessary.
>>
>> I don't think it's quite that, it's more that it's very hard to do.
>> Harman famously made a machine that could quickly exchange speakers so
>> that they could be compared the same position. With an opaque cloth,
>> this removed the physical appearance of the speakers from the
>> comparison so that the speakers could be evaluated by sound alone.
>> Audio reviewers could do the same.
>

> So that the speakers could be in the same position? The problem with
> this seems obvious.

The advantage of this technique is that it makes it possible for the
speakers to be in the same position, and it allows them to be
exchanged quickly. This allows very short-term auditory memory to be
used. I can't see any disadvantage: it's not as if you're forced to
have them in the same position,

Andrew.

[Arny Krueger]

"Scott" <S888...@aol.com> wrote in message
news:kai7a...@news4.newsguy.com...

On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:

>
>> The nice thing about testing for difference as ABX does is that if there
>> is no difference detected you know that the more expensive one is not any
>> better sounding. Unless it has features you > feel you must have or
>> you just like the look
>> better you can save some money. Personally, I like knowing that a
>> $2000.00 set of electronics is not going to be out performed by a
>> $20,000.00 set. > Speakers of course, (the part that you actually hear in
>> a sound system)
>> are another story entirely.

> heck if it makes you feel better about buying less expensive gear I guess
> that's nice.

That comment seems to be descending a steeply downward angled nose. ;-)

> But you are putting way too much weight on such a test if you think you
> walk away from a single null result "knowing"
> that the more expensive gear is not better sounding.

Ignores the fact that we are repeatedly told that hyper-expensive equipment
sounds "mind blowingly" better and that one has to be utterly tasteless to
not notice the difference immediately.

Also ignores the fact that all known objective bench testing and its
interpretation in conjunction with our best and most recent knowlege of
psychoacoustics says that no audible differences can be reasonably be
expected to be heard.

> But hey, if it makes you happy that's great.

It makes me happy to know that the best available current science actually
works out in the real world and that technological progress is still taking
place.

It makes me happy that good sound can be available to the masses if they
throw off the chains of tradition and ignorance.

I am also happy to see recognition of the fact that simply throwing vast
piles of money at solving problems that have been solved for a long time
doesn't help solve them. If we could only convince our politicians of that!
;-)

> But not everyone is on board with you there.

Exactly. Those who have invested heavily in anti-science probably did so
because they are in some state of being poorly informed or are in denial of
the relevant scientific facts. There can be very little rational that can be
said to change their minds because rational thought has nothing to do with
what they currently believe.

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kaknd...@news6.newsguy.com...

On Saturday, December 15, 2012 8:08:32 AM UTC-8, Scott wrote:

> My sentiments exactly. I'm convinced that while DBTs work great for drug
> tests, tests by food manufacturers about new or altered products, etc.,
> I'm not terribly sure that they work for audio equipment because the
> waveform that we are "analyzing" with our collective ears is pretty
> complex.

Anybody who has seen how certain tightly held but anti-scientific beliefs
are readily deconstructed using the results of bias-controlled listening
tests can see how people who keep on holding onto those beliefs would have
reservations about such a clear source of evidence that disagrees with them.

[quote]

Otherwise for DACs, preamps and amps, there are certainly differences (in
DACs, especially) yet they don't show-up in DBTs and ABX tests.

[/quote]

On balance we have a world that is full of DACs with better than +/- 0.1 dB
frequency response over the actual audible range and > 100 dB dynamic range.
They now show up in < $100 music players and < $200 5.1 channel AVRs. Where
in fact are the audible differences in those DACs supposed to be coming
from?

[quote]

Granted, with modern, solid-state amps and preamps the differences are
minute (and largely inconsequential), but they do show themselves in
properly set up DBT tests.

[/quote]

No adequate documentation of the above alleged fact has been seen around
here AFAIK.

[Arny Krueger]

"Scott" <S888...@aol.com> wrote in message

news:kai79...@news4.newsguy.com...

On Dec 14, 8:21 pm, Audio_Empire <gmgrav...@comcast.net> wrote:

> The person who was questioning to value of level matching did not seem
> to be limiting his opinion to CDPs and amps.

Seems like the backwards side of the argument. Doing comparisons of music
players, DACs and amps without proper level matching seems to be the prelude
to a massive waste of time. If the levels are not matched well enough then
there will be audible differences, but we have no way of knowing that the
causes are not our poor testing practices as opposed to any relevent
property of the equipment being tested.

> You still have the same
> problems in level matching that I stated above when dealing with
> loudspeakers. In fact you have even more problems with radiation
> pattern differences and room interfaces that make it even more
> impossible to do a true level match.

The known technical differences among loudspeakers are immense and gross
compared to those among music players, DACs and amps. I know of nobody who
claims that speakers can be sonically indistinguishable except in limited,
trivial cases. I don't know how this fact relates to a thread about "A brief
history of CD DBT" except as a distraction or red herring argument.

[Audio_Empire]

I was once in a stereo store in London England where they had two
turntables (speaker Lazy-Susans?), half of which were hidden by a
false wall, so that as the turntables turned, speakers (R & L) would
emerge from behind the false wall, and the turntable would stop
with the pair of speakers side-by-side about 6 ft apart and they
would be connected to the amplifier automatically. If you wanted
hear another pair, the sales guy would push a button and the two
turntables would turn and another pair would emerge and then
lock into place. How they hooked them up and automatically
changed the connections from speaker set to speaker set as
each came into position, I can only guess. They must have had
some kind of commutator arrangement on the underside of the
two "Lazy Susans." I have never seen it done that way anywhere
else - and I'm not sure a high-end store would want to do it
that way for fear that the commutator arrangement would
compromise the sound by virtue of introducing a set of
contacts between the amplifier and the speakers.

[Audio_Empire]

On Monday, December 17, 2012 6:48:54 AM UTC-8, Arny Krueger wrote:
> "Audio_Empire" <gmgr...@comcast.net> wrote in message
>
> news:kaknd...@news6.newsguy.com...
>
> On Saturday, December 15, 2012 8:08:32 AM UTC-8, Scott wrote:
>
>
>
> > My sentiments exactly. I'm convinced that while DBTs work great for drug
>
> > tests, tests by food manufacturers about new or altered products, etc.,
>
> > I'm not terribly sure that they work for audio equipment because the
>
> > waveform that we are "analyzing" with our collective ears is pretty
>
> > complex.
>
>
>
> Anybody who has seen how certain tightly held but anti-scientific beliefs
>
> are readily deconstructed using the results of bias-controlled listening
>
> tests can see how people who keep on holding onto those beliefs would have
>
> reservations about such a clear source of evidence that disagrees with them.

Well, first of all, Those "beliefs" that are are saddling me with are not "anti-scientific".
There are differences in electronic equipment and I'm convinced that some day there
will be tests that will reveal them. I've been in electronics long enough to know that
you will never uncover a piece of gear's flaws if your suit of measurements keep
measuring the wrong thing. Unfortunately, I don't know (any more than anyone else)
what we would test to account for the differences in modern amps (very small
differences, probably not worth the effort) or DACs (much larger differences). None
of these things are addressed in any test suite I've seen. Yes, we measure frequency
response, IM and harmonic distortion, channel separation, impulse response (in
DACs) perhaps we use an oscilloscope to look at square waves to measure low and
high frequency phase shift, but none of those really address things like the difference
between the imaging ability of two DACs, for instance, Where one of them has a more
three-dimensional image presentation that the other especially since both DACs
measure similar channel separation (which is so high in digital that as to ostensibly
be, for all practical purposes, beyond the limits of the human ear to perceive that
kind of isolation of right and left). Obviously, there is something that we humans
are not measuring.

>
>
>
> [quote]
>
> Otherwise for DACs, preamps and amps, there are certainly differences (in
>
> DACs, especially) yet they don't show-up in DBTs and ABX tests.
>
> [/quote]
>
>
>
> On balance we have a world that is full of DACs with better than +/- 0.1 dB
>
> frequency response over the actual audible range and > 100 dB dynamic range.
>
> They now show up in < $100 music players and < $200 5.1 channel AVRs. Where
>
> in fact are the audible differences in those DACs supposed to be coming
>
> from?

That's the puzzlement isn't it? Like I said, if the accepted suite of audio measurements
don't answer the questions, then obviously there is something that we don't measure.
It's the only plausible answer (and don't posture that these differences are imaginary;
the product of listening biases, because they aren't.

> [quote]
>
> Granted, with modern, solid-state amps and preamps the differences are
>
> minute (and largely inconsequential), but they do show themselves in
>
> properly set up DBT tests.
>
> [/quote]
>
>
>
> No adequate documentation of the above alleged fact has been seen around
> here AFAIK.

I agree. It's a puzzlement. I know that I, and several other audio enthusiasts of
my acquaintance can tell the difference between two amps in a carefully set-up
DBT almost every time. Yet. others in these ad-hoc tests seem to hear no differences,
and their results are essentially random, I.E. a null result. The only thing that I can
come up with is that I have been listening critically to different components for so
long, that I pick-up on audible clues that others simply miss.

[Dick Pierce]

Audio_Empire wrote:
> That's the puzzlement isn't it? Like I said, if the accepted suite of audio measurements
> don't answer the questions, then obviously there is something that we don't measure.
> It's the only plausible answer (and don't posture that these differences are imaginary;
> the product of listening biases, because they aren't.

But you have, in effect, stated elsewhere in this thread that
as far as you are concerned, the only "accepted suite of audio
measurements" for, say, power amplifiers is power, frequency
response, and distortion, yet, for decades, far more has not
only been available, it has been routinnely used.

Some of the measurements, for example, TIM, have been shown to
be irrelevant because they force conditions that are so utterly
unrealistic that the tell us nothing at all about the performance
of systems under conditions of listening to signal, oh, like music.
Other, like "damping factor" have been shown to not only be
irrelevant, but useless, except in the most pathological of
cases.

Some other measurements, like multi-tone intermodulation, may have
more relevance.

However, what we see hashed over and over again are manufacturers
specification masquerading as "measurements." Fine, we all agree they
are not the same. So why do we see them trotted out time and again,
erected as a strawman do be knocked down, and for what purpose?

If you want to talk about measurements, fine. do so.

But the "accepted quite" of audio measurements in the high-end
audio world, is QUITE different than the "accepted suite" of
audio measurements in a much bigger, richer and certainly much
more informed world than the tiny clique of high-end audio
affords.

--
+--------------------------------+
+ Dick Pierce |
+ Professional Audio Development |
+--------------------------------+

[nab...@hotmail.com]

On Monday, December 17, 2012 5:41:27 PM UTC-5, Audio_Empire wrote:

> Well, first of all, Those "beliefs" that are are saddling me with are not "anti-scientific".
> There are differences in electronic equipment and I'm convinced that some day there
> will be tests that will reveal them. I've been in electronics long enough to know that
> you will never uncover a piece of gear's flaws if your suit of measurements keep
> measuring the wrong thing. Unfortunately, I don't know (any more than anyone else)
> what we would test to account for the differences in modern amps (very small
> differences, probably not worth the effort) or DACs (much larger differences). None
> of these things are addressed in any test suite I've seen. Yes, we measure frequency
> response, IM and harmonic distortion, channel separation, impulse response (in
> DACs) perhaps we use an oscilloscope to look at square waves to measure low and
> high frequency phase shift, but none of those really address things like the difference
> between the imaging ability of two DACs, for instance, Where one of them has a more
> three-dimensional image presentation that the other especially since both DACs
> measure similar channel separation (which is so high in digital that as to ostensibly
> be, for all practical purposes, beyond the limits of the human ear to perceive that
> kind of isolation of right and left). Obviously, there is something that we humans
> are not measuring.

This is not obvious at all. First, amps and DACs are not mysteries of nature; they are man-made objects. If we couldn't measure their performance, we could not design them in the first place. I'm fairly certain that the poster here does not know how to design audio gear, so perhaps it is all magic to him. That would explain his viewpoint.

Second, there really isn't that much to measure. An audio signal, like all electrical signals, has only two attributes: amplitude and frequency. (Note that an eardrum's movement has the same two attributes.) We can be fairly certain that we are measuring amplitude and frequency quite accurately. There's really nothing missing.

Finally, what seals the case is that our two methods of assessing audibility--measurements and DBTs--agree with each other. That's how science validates itself--by finding multiple confirmations of the same conclusions. If AE were right, then BOTH our measurements AND our listening tests would have to be flawed, and flawed in the same way. That would be a very strange thing, given that they were developed independently.

bob

[Scott]

On Dec 17, 6:43 am, "Arny Krueger" <ar...@wowway.com> wrote:
> "Scott" <S888Wh...@aol.com> wrote in message

>
> news:kai7a...@news4.newsguy.com...
> On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:
>
>
>
> >> The nice thing about testing for difference as ABX does is that if there
> >> is no difference detected you know that the more expensive one is not any
> >> better sounding. Unless it has features you     > feel you must have or
> >> you just like the look
> >> better you can save some money. Personally, I like knowing that a
> >> $2000.00 set of electronics is not going to be out performed by a
> >> $20,000.00 set. > Speakers of course, (the part that you actually hear in
> >> a sound system)
> >> are another story entirely.
> > heck if it makes you feel better about buying less expensive gear I guess
> > that's nice.
>
> That comment seems to be descending a steeply downward angled nose. ;-)

Quite the contrary. I am actually happy to see people enjoying their
hobby. It is in the end a perceptual based en devour. If believing
everything sounds the same makes one happy that is great. If believing
wrapping the cat in tin foil and freezing pictures of your grandma
makes your system sound better that is great to. Unless you are the
cat. But misrepresenting science is not OK. I do take issue with
that.
.

>
> >  But you are putting way too much weight on such a test if you think you
> > walk away from a single null result "knowing"
> > that the more expensive gear is not better sounding.
>
> Ignores the fact that we are repeatedly told that hyper-expensive equipment
> sounds "mind blowingly" better and that one has to be utterly tasteless to
> not notice the difference immediately.

And here is a classic case in point. You are getting ready to wave the
science flag again in this post and here you are suggesting that a
proper analysis of data would include taking audiophile banter into
account. Understanding he true significance of a single null result
does not require consideration of you or anyone else has been told by
other audiophiles. For that to affect the weight placed on any single
test result would quite unscientific thinking.

>
> Also ignores the fact that all known objective bench testing and its
> interpretation in conjunction with our best and most recent knowlege of
> psychoacoustics says that no audible differences can be reasonably be
> expected to be heard.

And here we have a gross misrepresentation of the facts.

>
> >  But hey, if it  makes you happy that's great.
>
> It makes me happy to know that the best available current science actually
> works out in the real world and that technological progress is still taking
> place.

Makes me happy too. Not sure what has to do with my post though. I
suppose indirectly we should both be happy that the best available
current science is built on a rigorous execution of the scientific
method and an understanding of the weight that should be given to any
single result of any given piece of research. It makes me happy that
real scientists know better than to ever make claims of fact based on
a single null result.

>
> It makes me happy that good sound can be available to the masses if they
> throw off the chains of tradition and ignorance.

So it's a good thing that Stereo Review is dead then. :-)

> > But not everyone is on board with you there.
>
> Exactly. Those who have invested heavily in anti-science probably did so
> because they are in some state of being poorly informed or are in denial of
> the relevant scientific facts. There can be very little rational that can be
> said to change their minds because rational thought has nothing to do with
> what they currently believe.

And there you go waving that science flag again. It's OK as far as I
am concerned that you believe whatever you want to believe about
audio. But I will continue to call you out on your constant
misrepresentations of real science.

[Scott]

On Dec 17, 6:49 am, "Arny Krueger" <ar...@wowway.com> wrote:
> "Scott" <S888Wh...@aol.com> wrote in message

>
> news:kai79...@news4.newsguy.com...
> On Dec 14, 8:21 pm, Audio_Empire <gmgrav...@comcast.net> wrote:
>
> > The person who was questioning to value of level matching did not seem
> > to be limiting his opinion to CDPs and amps.
>
> Seems like the backwards side of the argument. Doing comparisons of music
> players, DACs and amps without proper level matching seems to be the prelude
> to a massive waste of time. If the levels are not matched well enough then
> there will be audible differences, but we have no way of knowing that the
> causes are not our poor testing practices as opposed to any relevent
> property of the equipment being tested.

Why on earth would you cut out all the relevant discussion and then
post the obvious which has already been covered? I already stated that
level matching is essential in any ABX DBT of the above components
since the goal of an ABX test is only to test for audible differences
and not preferences

.
>
> > You still have the same
> > problems in level matching that I stated above when dealing with
> > loudspeakers. In fact you have even more problems with radiation
> > pattern differences and room interfaces that make it even more
> > impossible to do a true level match.
>
> The known technical differences among loudspeakers are immense and gross
> compared to those among music players, DACs and amps. I know of nobody who
> claims that speakers can be sonically indistinguishable except in limited,
> trivial cases. I don't know how this fact relates to a thread about "A brief
> history of CD DBT" except as a distraction or red herring argument.

I suggest you follow the thread more closely if you think this is a
red herring argument rather than a relevant point in regards to issues
raised in this thread by another poster when it comes to use of DBTs
for determining preferences And the relative merits and difficulties
of level matching that one has to deal with in doing blind preference
comparison tests with things that really cant be truly level matched
due to differences in dynamic range and frequency response among other
things.

[KH]

On 12/17/2012 8:46 PM, Scott wrote:
> On Dec 17, 6:43 am, "Arny Krueger" <ar...@wowway.com> wrote:
>> "Scott" <S888Wh...@aol.com> wrote in message
>>
>> news:kai7a...@news4.newsguy.com...
>> On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:
>>

<snip>

>>
>>> But you are putting way too much weight on such a test if you think you
>>> walk away from a single null result "knowing"
>>> that the more expensive gear is not better sounding.
>>
>> Ignores the fact that we are repeatedly told that hyper-expensive equipment
>> sounds "mind blowingly" better and that one has to be utterly tasteless to
>> not notice the difference immediately.
>
> And here is a classic case in point. You are getting ready to wave the
> science flag again in this post and here you are suggesting that a
> proper analysis of data would include taking audiophile banter into
> account.

In this instance, as Arny presented it, it would not be "banter", but
would, rather, define the null hypothesis. I.e., instead of being
"there are no audible differences", it becomes "there are no major,
unmistakeable audible differences". In a "typical" audiophile scenario,
these are the differences described. How many of these claims are
"unmistakeable", "not at all subtle" etc. In constructing the null
hypothesis of any test these qualifiers cannot be casually ignored.

This is, to me, the heart of the stereotypical subjectivist argument
against DBT or ABX testing - the differences are claimed as obvious
sighted, but then become obscured by any imposed test rigor. In testing
any such claim, the magnitude of the difference (e.g. "obvious to anyone
with ears") defines the precision and detectability requirements of the
test design.

> Understanding he true significance of a single null result
> does not require consideration of you or anyone else has been told by
> other audiophiles.

That would rest entirely upon how the null hypothesis is constructed,
and may indeed include such claims.

>For that to affect the weight placed on any single
> test result would quite unscientific thinking.

Again, simply not accurate with respect to the world of possible
hypotheses. Any null result for a discrimination test evaluating
"obvious" differences will be significant, if not dispository, for that
test and equipment, as long as the test is set up properly.

>>
>> Also ignores the fact that all known objective bench testing and its
>> interpretation in conjunction with our best and most recent knowlege of
>> psychoacoustics says that no audible differences can be reasonably be
>> expected to be heard.
>
> And here we have a gross misrepresentation of the facts.
>
>
>>
>>> But hey, if it makes you happy that's great.
>>
>> It makes me happy to know that the best available current science actually
>> works out in the real world and that technological progress is still taking
>> place.
>
> Makes me happy too. Not sure what has to do with my post though. I
> suppose indirectly we should both be happy that the best available
> current science is built on a rigorous execution of the scientific
> method and an understanding of the weight that should be given to any
> single result of any given piece of research. It makes me happy that
> real scientists know better than to ever make claims of fact based on
> a single null result.
>

Sorry, but you seem to be using a rather unique definition of "fact" as
"real scientists" make claims of fact for every such study. The results
*are* facts, and are true, and applicable, within the constraints and
confidence interval of the test design. To believe otherwise would
require a refutation of statistics. If you doubt this, then please
explain exactly how many tests are required to result in "facts".

Keith

[Dick Pierce]

nab...@hotmail.com wrote:
>
> Second, there really isn't that much to measure. An audio signal,
> like all electrical signals, has only two attributes: amplitude and
> frequency.

Actually, to be moe precise, the two fundamental attributes are
ampliude and time, but your point remains: it's not that it's
some higher-order dimensional thingy with some of those dimensions
hidden. From the amplitude vs time signal we can derive other
information such as the amplitude vs frequency your mention.
Two ways come to mind of doing this: through various mathematical
transforms or through a process called hearing.

Extended to a DAC or a typical power amplifer, this two-dimensional
pr0oblem becomes a three-dimensional one: the amplitude of the right
channel and the amplitude of the left channel vs time (we assume the
two channel use the same time :-). There's nothing else going on,
there is no other "hidden channel" for "hidden information."

But NONE of this is a mystery, at least not to those in the signal
processing, acoustical/psychophysical realm.

It may well be a mystery (and often times seems like it is) to those
in high-end audio. But, I'd assert, that's an education problem.

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kao72...@news3.newsguy.com...

> There are differences in electronic equipment and I'm convinced that some
> day there
> will be tests that will reveal them.

That the equpment is different is fact. The question at hand is not about
that fact. The question at hand is about the audible significance of those
differences.

The use of audio gear seems to be pretty straight-forward and simple. We
apply audio signals to audio gear, turn it into sound in listening rooms
using loudspeakers and headphones, and listen to it.

The symmetry between listening tests and listening to music for enjoyment
can be as complete as we have the patience to make it so.

It is ironic to me that much of the so-called evidence supporting the
existence of mysterious equipment properties that elude sophisticated
testing is obtained by such crude means. It even eludes all known attempts
to duplicate the crude means while imposing basic a few basic, simple bias
controls by the least intrusive means found after extensive investigation
and experimentation.

If you are talking about technical tests then the solution to our problem
can be found in multivariate calculus. It is a mathematical fact that any
system with a finite dimensional state can be fully analyzed. An audio
channel has two variables being time and intensity. It is very simple.
Mathematicians have analyzed these two variables for maybe 100 years
(analysis acutally started no less recently than with Fourier).

> I've been in electronics long enough to know that
> you will never uncover a piece of gear's flaws if your suit of
> measurements keep
> measuring the wrong thing.

That's a truism, but without more specifics it is just idle speculation.

> Unfortunately, I don't know (any more than anyone else)
> what we would test to account for the differences in modern amps (very
> small
> differences, probably not worth the effort) or DACs (much larger
> differences).

What differences are we testing for - things that only show up in sighted
evaluations or evaluations that are semi-, demi-, or quasi controlled?

Once we learned how to do reliable listening tests back in the 1970s there
have been no mysteries - what we hear we measure and vice versa given that
we measure enough to be audible.

As others have pointed out one of the first casualties of reliable listening
tests was the hysteria over slew rate induced distoriton.

> None of these things are addressed in any test suite I've seen.

None of what? So far I see no actual description of something with hands and
feet.

> Yes, we measure frequency
> response, IM and harmonic distortion, channel separation, impulse response
> (in
> DACs) perhaps we use an oscilloscope to look at square waves to measure
> low and
> high frequency phase shift, but none of those really address things like
> the difference
> between the imaging ability of two DACs, for instance,

Yet another audiophile myth that dies a quick death when you start doing
adequately controlled listening tests.

[Scott]

Well thank goodness in real science researchers know better than to
move the goal posts due to trash talking between audiophiles. I would
think that if objectivists were genuinely interested in applying
science to the question of amplifier sound they would not move the
goal posts nor would they use ABX DBTs the way they have when it comes
to amplifier sound. That being typically breaking out ABX and failing
to ever control for same sound bias or even calibrate the sensitivity
of the test. Without such calibration a single null result tells us
very little about what was and was not learned about the sound of the
components under test.

But of course my point was the fact that no scientist worth his or her
salt would ever make dogmatic claims of fact based on the results of
any single ABX DBT null. And if one think that claims from
subjectivists should alter that fact then they simply don't understand
how real science deals with and interprets real scientific data.

>
> > Understanding he true significance of a single null result
> > does not require consideration of you or anyone else has been told by
> > other audiophiles.
>
> That would rest entirely upon how the null hypothesis is constructed,
> and may indeed include such claims.

No it does not. Real science builds it's conclusions on an
accumulation of research. Again if one understands how science works
they should know the real standing of one singular null result. That
being it is most certainly not something one can reasonably close the
books on and say that it is final proof of no difference.

>
> >For that to affect the weight placed on any single
> > test result would quite unscientific thinking.
>
> Again, simply not accurate with respect to the world of possible
> hypotheses.  Any null result for a discrimination test evaluating
> "obvious" differences will be significant, if not dispository, for that
> test and equipment, as long as the test is set up properly.

Sorry but you are plainly wrong. No scientist would ever put that much
stock in one test. It runs contrary to the very idea of
falsifiability, peer review or the idea of verification via repetition
of previous tests.very very unscientific

>
>
>
>
>
>
>
>
>
>
>
> >> Also ignores the fact that all known objective bench testing and its
> >> interpretation in conjunction with our best and most recent knowlege of
> >> psychoacoustics says that no audible differences can be reasonably be
> >> expected to be heard.
>
> > And here we have a gross misrepresentation of the facts.
>
> >>>   But hey, if it  makes you happy that's great.
>
> >> It makes me happy to know that the best available current science actually
> >> works out in the real world and that technological progress is still taking
> >> place.
>
> > Makes me happy too. Not sure what has to do with my post though. I
> > suppose indirectly we should both be happy that the best available
> > current science is built on a rigorous execution of the scientific
> > method and an understanding of the weight that should be given to any
> > single result of any given piece of research. It makes me happy that
> > real scientists know better than to ever make claims of fact based on
> > a single null result.
>
> Sorry, but you seem to be using a rather unique definition of "fact" as
> "real scientists" make claims of fact for every such study.

Complete nonsense. And you say this after bring up the null
hypothesis. You might want to read up on the null hypothesis and what
it proves and what it does not prove.
http://en.wikipedia.org/wiki/Null_hypothesis

>  The results
> *are* facts, and are true, and applicable, within the constraints and
> confidence interval of the test design.  To believe otherwise would
> require a refutation of statistics.  If you doubt this, then please
> explain exactly how many tests are required to result in "facts".

No the results are not facts the results are data. Often in this kind
of research one will find conflicting data. That is what no one who
understands these kinds of things would ever draw a conclusion of fact
from a single test. To say it would be a hasty conclusion would be an
understatement.

[Arny Krueger]

"Scott" <S888...@aol.com> wrote in message

news:ajbmr5...@mid.individual.net...

> Well thank goodness in real science researchers know better than to
> move the goal posts due to trash talking between audiophiles. I would
> think that if objectivists were genuinely interested in applying
> science to the question of amplifier sound they would not move the
> goal posts nor would they use ABX DBTs the way they have when it comes
> to amplifier sound.

The last above seems to show considerable bias. It seems to say that no
"true scientist" has ever done an ABX test of an amplfiier.

> That being typically breaking out ABX and failing
> to ever control for same sound bias

I've done some checking and the phrase "sound bias" appears to a a
contrivance of its author. It has no standard defined meaning that I know of
or can find in the literature of audio.

> or even calibrate the sensitivity of the test.

I've explained the many ways the the results of blind listening tests to
date have been confirmed, calibrated double and triple checked on RAHE many
times. Amnesia?

> But of course my point was the fact that no scientist worth his or her
> salt would ever make dogmatic claims of fact based on the results of
> any single ABX DBT null.

At this point 100s if not 1,000s of ABX tests have been performed so the
claim that any claims dogmatic or otherwise would be based on the results of
far more than just one test. Straw man argument.

[nab...@hotmail.com]

On Tuesday, December 18, 2012 9:35:07 AM UTC-5, Dick Pierce wrote:
>
> > Second, there really isn't that much to measure. An audio signal,
> > like all electrical signals, has only two attributes: amplitude and
> > frequency.
>
> Actually, to be moe precise, the two fundamental attributes are
> ampliude and time,

Granted, but to us non-scientists, it is easier to think in terms of amplitude and frequency, because they correspond to concepts we are readily familiar with (i.e., loudness and pitch).

<snip>

> But NONE of this is a mystery, at least not to those in the signal
> processing, acoustical/psychophysical realm.>
>
> It may well be a mystery (and often times seems like it is) to those
> in high-end audio. But, I'd assert, that's an education problem.

Miseducation is precisely the problem. And the audiophile rags have a lot to answer for on that score.

bob

[Scott]

On Dec 18, 11:48 am, "Arny Krueger" <ar...@wowway.com> wrote:
> "Scott" <S888Wh...@aol.com> wrote in message

>
> news:ajbmr5...@mid.individual.net...
>
> > Well thank goodness in real science researchers know better than to
> > move the goal posts due to trash talking between audiophiles. I would
> > think that if objectivists were genuinely interested in applying
> > science to the question of amplifier sound they would not move the
> > goal posts nor would they use ABX DBTs the way they have when it comes
> > to amplifier sound.
>
> The last above seems to show considerable bias.

In your biased opinion :-)

> It seems to say that no
> "true scientist" has ever done an ABX test of an amplfiier.

That is an odd interpretation. It certainly is not what i was saying
at all. There are many scientists in this world any number of whom are
audiophiles. I would hardly make any claim that none of them have ever
done an ABX test of an amplifier. i have no idea what every scientist
is doing in their spare time for fun. OTOH I have yet to see a peer
reviewed paper published in any scientific journal of ABX tests done
on amplifiers. I would hope though, that any such scientific journal
would call out any such paper should it be shown that the participants
knew in advance what amplifiers A and B were and nothing was done to
control for a possible same sound bias and nothing was done to
demonstrate the test was actually sensitive to real and subtle audible
differences should the result be a null. So if there are such studies
that you know of please cite them. I would be very interested in
reading them.

>
> >  That being typically breaking out ABX and failing
> > to ever control for same sound bias
>
> I've done some checking and the phrase "sound bias" appears to a a
> contrivance of its author. It has no standard defined meaning that I know of
> or can find in the literature of audio.

My goodness. if you cut the phrase in the middle what do you expect?
OTOH you could talk to your friend JJ Johnston on the need or lack
there of for positive and negative controls in DBTs. More specifically
you might ask him if he thinks "same sound bias" is not an issue in an
ABX test if the subject knows in advance what A and B are. Go ahead,
ask him. ;-) I don't think you are going to like the answer....

>
> >  or even calibrate the sensitivity of the test.
>
> I've explained the many ways the the results of blind listening tests to
> date have been confirmed, calibrated double and triple checked on RAHE many
> times. Amnesia?

Please cite how the Stereophile Tests we have been debating calibrated
the test for sensitivity to audible differences.

>
> > But of course my point was the fact that no scientist worth his or her
> > salt would ever make dogmatic claims of fact based on the results of
> > any single ABX DBT null.
>
> At this point 100s if not 1,000s of ABX tests have been performed so the
> claim that any claims dogmatic or otherwise would be based on the results of
> far more than just one test. Straw man argument.

Not interested in cherry picked anecdotal evidence from flawed tests.

[nab...@hotmail.com]

On Tuesday, December 18, 2012 12:18:29 PM UTC-5, Scott wrote:

> That being typically breaking out ABX and failing
> to ever control for same sound bias

There is no such phenomenon as same sound bias. It has never been demonstrated experimentally. If you have data that shows otherwise, please share it with us.

> or even calibrate the sensitivity
> of the test. Without such calibration a single null result tells us
> very little about what was and was not learned about the sound of the
> components under test.

There is no need to "calibrate the sensitivity" of an ABX test of audio components, anymore than there is a need to calibrate the sensitivity of a DB pharmaceutical trial. In both cases, we care only about subjects' sensitivity to a given dose (or in the case of ABX, a given difference). We aren't trying to determine the minimum dose/difference the subjects might respond to.

Regardless, of course a single null result tells us very little. But my original post did not present a single result. It presented a substantial number of tests (not all null, btw) conducted over a long period of time by widely disparate groups.

bob

[Audio_Empire]

On Monday, December 17, 2012 6:43:52 AM UTC-8, Arny Krueger wrote:
> "Scott" <S888...@aol.com> wrote in message
>
> news:kai7a...@news4.newsguy.com...
>
> On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:
>
>
>
> >
>
> >> The nice thing about testing for difference as ABX does is that if there
>
> >> is no difference detected you know that the more expensive one is not any
>
> >> better sounding. Unless it has features you > feel you must have or
>
> >> you just like the look
>
> >> better you can save some money. Personally, I like knowing that a
>
> >> $2000.00 set of electronics is not going to be out performed by a
>
> >> $20,000.00 set. > Speakers of course, (the part that you actually hear in
>
> >> a sound system)
>
> >> are another story entirely.
>
>
>
> > heck if it makes you feel better about buying less expensive gear I guess
>
> > that's nice.
>
>
>
> That comment seems to be descending a steeply downward angled nose. ;-)
>
>
>
> > But you are putting way too much weight on such a test if you think you
>
> > walk away from a single null result "knowing"
>
> > that the more expensive gear is not better sounding.
>
>
>
> Ignores the fact that we are repeatedly told that hyper-expensive equipment
>
> sounds "mind blowingly" better and that one has to be utterly tasteless to
>
> not notice the difference immediately.

But your "hyper-expensive" gear is not so "mind blowingly" better than the less
expensive gear. It is subtly different, usually marginally cleaner, especially in the
top-end where the highs are less "grainy" and smoother than more run-of-the-
mill components. But many people cannot (or will not) hear the differences.
That's not their fault, really. Honest high-end equipment manufacturers use
the best quality components in their audio gear. They use the best capacitors,
the least noisy resistors, the finest potentiometers, the best grade of switches,
etc. This accounts for SOME of the high prices that these devices demand. And
it usually results in slightly better sound. But as I have stated before, while these
differences do exist, they are so small that if one bought any of them and inserted
them into their systems, after an hour of listening, they would find nothing to
complain about and happily accept the sound they are getting, forgetting any
of the differences that they may have heard in a DBT "shoot-out" between the
amps in question.

>
>
>
> Also ignores the fact that all known objective bench testing and its
>
> interpretation in conjunction with our best and most recent knowlege of
>
> psychoacoustics says that no audible differences can be reasonably be
>
> expected to be heard.

Then somebody is measuring the wrong thing.

> > But hey, if it makes you happy that's great.
>
>
>
> It makes me happy to know that the best available current science actually
>
> works out in the real world and that technological progress is still taking
>
> place.
>
>
>
> It makes me happy that good sound can be available to the masses if they
>
> throw off the chains of tradition and ignorance.
>
>
>
> I am also happy to see recognition of the fact that simply throwing vast
>
> piles of money at solving problems that have been solved for a long time
>
> doesn't help solve them. If we could only convince our politicians of that!
>
> ;-)
>
>
>
> > But not everyone is on board with you there.
>
>
>
> Exactly. Those who have invested heavily in anti-science probably did so
>
> because they are in some state of being poorly informed or are in denial of
>
> the relevant scientific facts. There can be very little rational that can be
>
> said to change their minds because rational thought has nothing to do with
>
> what they currently believe.

And those who have invested heavily in the notion that current science has all
the answers, can pinch their pennies and enjoy lesser equipment safe in their
delusion that It's all the same....

[Audio_Empire]

On Monday, December 17, 2012 6:07:17 PM UTC-8, Dick Pierce wrote:
> Audio_Empire wrote:
>
> > That's the puzzlement isn't it? Like I said, if the accepted suite of audio measurements
>
> > don't answer the questions, then obviously there is something that we don't measure.
>
> > It's the only plausible answer (and don't posture that these differences are imaginary;
>
> > the product of listening biases, because they aren't.
>
>
>
> But you have, in effect, stated elsewhere in this thread that
>
> as far as you are concerned, the only "accepted suite of audio
>
> measurements" for, say, power amplifiers is power, frequency
>
> response, and distortion, yet, for decades, far more has not
>
> only been available, it has been routinnely used.

If you "got" that from any of my posts, then may I recommend remedial
reading comprehension, or perhaps that you read with less interpretive
imagination. Because to my knowledge, I've not inferred or stated anything
of the kind. I did say that these are things that are most often quoted
in spec sheets, but I never said that it was enough just to measure these things.

>
>
>
> Some of the measurements, for example, TIM, have been shown to
>
> be irrelevant because they force conditions that are so utterly
>
> unrealistic that the tell us nothing at all about the performance
>
> of systems under conditions of listening to signal, oh, like music.
>
> Other, like "damping factor" have been shown to not only be
>
> irrelevant, but useless, except in the most pathological of
>
> cases.

Yes, that is true, but often they are still quoted as specs. That was
all I was inferring.

>
>
>
> Some other measurements, like multi-tone intermodulation, may have
>
> more relevance.
>
>
>
> However, what we see hashed over and over again are manufacturers
>
> specification masquerading as "measurements." Fine, we all agree they
>
> are not the same. So why do we see them trotted out time and again,
>
> erected as a strawman do be knocked down, and for what purpose?

Because most buyers are not technical and good specs are impressive,
perhaps? I don't pretend to know.

> If you want to talk about measurements, fine. do so.

I merely mentioned the suite of tests that is the most ubiquitous.
I don't particularly want to talk about them because generally
we don't know the circumstances under which many published
measurements are made. And without context, they can be misleading
and even meaningless.

>
>
>
> But the "accepted suite" of audio measurements in the high-end

>
> audio world, is QUITE different than the "accepted suite" of
>
> audio measurements in a much bigger, richer and certainly much
>
> more informed world than the tiny clique of high-end audio
>
> affords.

Yeah, but who cares? In the first place, Consumers don't understand them
and in the second place, most modern audio equipment measures so superlatively,
that any differences heard wouldn't correspond to those measurements
anyway.

[Audio_Empire]

On Tuesday, December 18, 2012 7:03:31 AM UTC-8, Arny Krueger wrote:
> "Audio_Empire" <gmgr...@comcast.net> wrote in message
>
> news:kao72...@news3.newsguy.com...
>
>
>
> > There are differences in electronic equipment and I'm convinced that some
>
> > day there
>
> > will be tests that will reveal them.
>
>
>
> That the equpment is different is fact. The question at hand is not about
>
> that fact. The question at hand is about the audible significance of those
>
> differences.
>
>
>
> The use of audio gear seems to be pretty straight-forward and simple. We
>
> apply audio signals to audio gear, turn it into sound in listening rooms
>
> using loudspeakers and headphones, and listen to it.

If it were that easy, there would perfect systems which would create
perfect facsimiles of the actual recorded event. I've never heard anyone
say that such-and-such a system was indistinguishable from the real
thing. Nor Have I ever heard anyone say that they mistook reproduced
music for live music. The most convincing I've ever heard was a pair
of the recent Wilson Alexandria XLF speakers driven by a pair of VTL
Siegfried II 800 Watt/channel power amps and a dCS Debussy CD
rig. It was good, I've never heard a pair of speakers load a room like
the XLFs. Impressive, but even with really good source material (like
my own recording of the Stanford University Jazz orchestra (made with
a single, big capsule stereo mike) it never fooled me into thinking that
it was anything but a very good recording, and the commercial stuff
they were demonstrating with in that hotel meeting room was even less
convincing. Si suggest that you revisit your statement that it's simple.

>
>
>
> The symmetry between listening tests and listening to music for enjoyment
>
> can be as complete as we have the patience to make it so.

That's true, but the word patience is the operative one here. Nobody
running these tests seem to take into account that soundstage
and imaging of say. a DAC, would be better served if the source
material actually had some real sound-stage engineered into it.
Frankly, due to the taste of most of the listeners involved in these tests,
good source material, material that would show things like
differences in soundstage presentation, generally aren't used. Also,
the people conducting such tests are so hung-up on instantaneous
A/B comparison that they never stick with one DUT or the other
long enough for the listening panel to focus-in on any differences
that aren't instantly recognizable.

> It is ironic to me that much of the so-called evidence supporting the
>
> existence of mysterious equipment properties that elude sophisticated
>
> testing is obtained by such crude means. It even eludes all known attempts
>
> to duplicate the crude means while imposing basic a few basic, simple bias
>
> controls by the least intrusive means found after extensive investigation
>
> and experimentation.

Ironic is it? I'd use another word, I think.

>
>
>
> If you are talking about technical tests then the solution to our problem
>
> can be found in multivariate calculus. It is a mathematical fact that any
>
> system with a finite dimensional state can be fully analyzed. An audio
>
> channel has two variables being time and intensity. It is very simple.
>
> Mathematicians have analyzed these two variables for maybe 100 years
>
> (analysis acutally started no less recently than with Fourier).

Like I said earlier, if it's so simple, how come NO stereo system, regardless
of price or acoustical setting can create a convincing facsimile of a real
performance playing in a real space? If you've ever been to New Orleans
and walked down Bourbon Street on a warm evening, you will have noticed,
as you walk down the sidewalk passing the open doors of this establishment
or that establishment, that you can tell in an instant, without even seeing the
source, in which establishments live music was playing, and in which
establishments the music is canned. The worlds finest stereo system, one
with state-of-the-art BIG speakers costing as much as a new Ferrari, simply
cannot convince anyone that the sound is real.

> > I've been in electronics long enough to know that
>
> > you will never uncover a piece of gear's flaws if your suit of
>
> > measurements keep
>
> > measuring the wrong thing.
>
>
>
> That's a truism, but without more specifics it is just idle speculation.

So "trisim" = speculation?

> > Unfortunately, I don't know (any more than anyone else)
>
> > what we would test to account for the differences in modern amps (very
>
> > small
>
> > differences, probably not worth the effort) or DACs (much larger
>
> > differences).
>
>
>
> What differences are we testing for - things that only show up in sighted
>
> evaluations or evaluations that are semi-, demi-, or quasi controlled?

Doesn't matter, but they do show up in carefully controlled tests as long as
the source material is of sufficient quality to allow these differences to be
heard, and as long as the testers aren't looking ONLY for differences that
reveal themselves in quick A/B comparisons.

>
>
>
> Once we learned how to do reliable listening tests back in the 1970s there
>
> have been no mysteries - what we hear we measure and vice versa given that
>
> we measure enough to be audible.



> As others have pointed out one of the first casualties of reliable listening
>
> tests was the hysteria over slew rate induced distoriton.
>
>
>
> > None of these things are addressed in any test suite I've seen.
>
>
>
> None of what? So far I see no actual description of something with hands and
>
> feet.

This is your selective editing at work, methinks.

> > Yes, we measure frequency
>
> > response, IM and harmonic distortion, channel separation, impulse response
>
> > (in
>
> > DACs) perhaps we use an oscilloscope to look at square waves to measure
>
> > low and
>
> > high frequency phase shift, but none of those really address things like
>
> > the difference
>
> > between the imaging ability of two DACs, for instance,
>
>
>
> Yet another audiophile myth that dies a quick death when you start doing
>
> adequately controlled listening tests.

That's just it. Mr. Kruger, IT DOESN'T die either a quick death or a slow one.
Also what it doesn't do is show up immediately on quick A/B tests. It also requires
that the recording used in the evaluation actually have some imaging specificity.
Most of the DBTs where I've been a listener, are pop, rock, and jazz recordings
which are studio creations, that, at the very best, are all multimiked, multi-
channel affairs and at worst either have no acoustic interments in them or,
have been Frapped! No imaging specificity there!

[nab...@hotmail.com]

On Tuesday, December 18, 2012 5:21:19 PM UTC-5, Audio_Empire wrote:

> Doesn't matter, but they do show up in carefully controlled tests as long as
> the source material is of sufficient quality to allow these differences to be
> heard, and as long as the testers aren't looking ONLY for differences that
> reveal themselves in quick A/B comparisons.

So where are these tests you keep talking about? I started this thread with a fairly healthy list of well-documented tests that show the opposite. You keep saying there are other tests that support your position, but you haven't presented even an iota of data to support this. Until you do . . .

bob

[Scott]

On Dec 18, 12:18 pm, nabo...@hotmail.com wrote:
> On Tuesday, December 18, 2012 12:18:29 PM UTC-5, Scott wrote:
> > That being typically breaking out ABX and failing
> > to ever control for same sound bias
>
> There is no such phenomenon as same sound bias.

That is plainly wrong. Biases come in all sorts of flavors including a
bias towards components sounding the same.

> It has never been demonstrated experimentally. If you have data that shows otherwise, please share it with us.

Well, I am so glad you asked. I have some pretty good data of one
clear cut example of same sound bias at work.
Let's take a trip down memory lane with Mr. Howard Ferstler and an
article he wrote for The Sensible Sound in which he dud an ABX DBT
between two amplifiers and concluded that it demonstrated the two
sounded the same. Let's look a little closer at what really went down

issue 88 of The $ensible Sound (Nov/Dec
2001, pp.10-17)

Howard wrote in his article on page 14

"According to the statistical analysis, and given the number of
trials I did, the likelihood of those scores being theresult of
anything but chance (even the one where I scored more than 60%right)
exceeded 95%." "Even though a 68% correct score looks like there may
have been significant audible differences with the 17 out of 25
mindnumbing trials I did, that score does achieve a 95% confidence
level, indicating that the the choices were still attributable to
chance."

John Atkinson point out to him the following facts.

“As has been pointed on this newsgroups, not only by myself but also
by Arny Krueger, you were misrepresenting the results, presumably
because they were "blatantly at odds with [your] belief systems." Yes,
scoring 17 out of 25 in a blind test does almost reach the 95%
confidence level (94.6%, to be pedantic). But this means that there is
almost 19 chances in 20 that you _did_ hear a difference between the
amplifiers. You incorrectly wrote in a published article that your
scoring 17 out of 25 was more than 95% due to chance. However it's
actually almost 95% not_ due to chance. In other words, your own tests
suggested you heard a difference, but as you already "knew" there
wasn't an audible difference, you drew the wrong conclusion from your
own data.
Curiously, The Sensible Sound has yet to publish a retraction. :-)”
John Atkinson
Editor, Stereophile

So we have here a classic example of same sound bias affecting the
analysis of the data of an ABX DBT between amps. But wait, it gets
better. Check out how Howard tries to reconcile his positive result
with his same sound bias.

Howard Ferstler:

" The data you are referring to was but a small part of the series.
It was a fluke, because during the last part of that series of trials
I was literally guessing. I just kept pushing the button and making
wild stabs at what I thought I heard. After a while, I did not bother
to listen at all. I just kept pressing the same choice over and
over."

IOW he was deliberately falsifying data in order to get a null result.
I’d say that is proof positive of a same sound bias on the part of Mr.
Ferstler wouldn’t you? And this ABX DBT was published in The Sensible
Sound despite the fact that the analysis was corrupted by a clear same
sound bias but so was the data, deliberately!
Ironically, due to an apparent malfunction in Tom Nousaine’s ABX box
the attempt at spiking the results to get a null serendipitously
wrought a false positive. So on top of that we have a mal functioning
ABX box that Tom Nousiane has been using for all these ABX DBTs.

Didn’t you at some point cite this very test and other tests conducted
with Tom Nousaine’s ABX box as "scientific evidence?"

Ouch.

>
> > or even calibrate the sensitivity
> > of the test. Without such calibration a single null result tells us
> > very little about what was and was not learned about the sound of the
> > components under test.
>
> There is no need to "calibrate the sensitivity" of an ABX test of audio components, anymore than there is a need to calibrate the sensitivity of a DB pharmaceutical trial.

My goodness gracious talk about getting it all wrong. First ABX DBTs
involves playback equipment. Pharaceutical trials do not so there is
nothing to "calibrate" in pharmeceutical trials. BUT they do use
control groups! That is in effect their calibration. without the
control group the results mean nothing because there is no
"calibrated" base to compare them to. So in effect they most
defintiely are calibrated or they are tossed out as very very bad
science and just plain junk. That is bias controlled testing 101.

> In both cases, we care only about subjects' sensitivity to a given dose (or in the case of ABX, a given difference). We aren't trying to determine the minimum dose/difference the subjects
> might respond to.

Wrong! In the pharmaceutical tests we don't care a bit about a
subject's sensitivity to a given dose. we care about the subject's
sensitivity as compared to the *control group* That is the
calibration!

>
> Regardless, of course a single null result tells us very little.

Gosh that is what i have been saying. So you agree. Great.

> But my original post did not present a single result. It presented a substantial number of tests (not all null, btw) conducted over a > long period of time by widely disparate groups.

And my comments about how it is very unscientific to put so much
weight in one null result was not a response to your original post.

However I do have to ask, did you include any of the tests by Howard
Ferstler? That would be most unfortunate. Did you include tests
conducted with Tom Nousaine's defective ABX box? That would also be
unfortunate. Funny what we learn when we dig a little. Such is the
point of peer review. To say that some the evidence presented in these
audio magazines is anecdotal is to be overly generous. That should be
obvious after what The Sensible Sound allowed to pass and be reported
as an ABX DBT of amplifiers.

[KH]

Well, some of us *are* engaged in *real* science on a daily basis, and
do understand the precepts.

> I would
> think that if objectivists were genuinely interested in applying
> science to the question of amplifier sound they would not move the
> goal posts nor would they use ABX DBTs the way they have when it comes
> to amplifier sound.

The thread has nothing to do with "amplifier" sound.

> That being typically breaking out ABX and failing
> to ever control for same sound bias or even calibrate the sensitivity
> of the test. Without such calibration a single null result tells us
> very little about what was and was not learned about the sound of the
> components under test.

Careful reading would show I clearly stipulated such requirements need
to be defined and accounted for. Arguing in favor of my stated position
isn't much of a refutation.

>
> But of course my point was the fact that no scientist worth his or her
> salt would ever make dogmatic claims of fact based on the results of
> any single ABX DBT null. And if one think that claims from
> subjectivists should alter that fact then they simply don't understand
> how real science deals with and interprets real scientific data.

The "dogmatic" claims, as you describe them, were based on physics and
engineering principles, and the fact that listening tests, under
controlled conditions, have not shown results that dispute those
principles. There was no claim, as I read it, that any individual test
was applicable to all conditions. Quite the opposite in fact - where
are the tests that contradict the the physics and engineering principles?

>>> Understanding he true significance of a single null result
>>> does not require consideration of you or anyone else has been told by
>>> other audiophiles.
>>
>> That would rest entirely upon how the null hypothesis is constructed,
>> and may indeed include such claims.
>
> No it does not. Real science builds it's conclusions on an
> accumulation of research.

No, every test has a conclusion, and is dispositive, if executed
accurately, within the limitations of the specific test.

> Again if one understands how science works
> they should know the real standing of one singular null result. That
> being it is most certainly not something one can reasonably close the
> books on and say that it is final proof of no difference.

The "books" are clearly closed on that test group, under those test
conditions. To think otherwise is to deny the relevance of all tests
under all conditions.

>>
>>> For that to affect the weight placed on any single
>>> test result would quite unscientific thinking.
>>
>> Again, simply not accurate with respect to the world of possible
>> hypotheses. Any null result for a discrimination test evaluating
>> "obvious" differences will be significant, if not dispository, for that
>> test and equipment, as long as the test is set up properly.
>
> Sorry but you are plainly wrong. No scientist would ever put that much
> stock in one test. It runs contrary to the very idea of
> falsifiability, peer review or the idea of verification via repetition
> of previous tests.very very unscientific

Nonsense. Do one tox study and argue that 90% severe adverse effects
doesn't mean anything. See how far that gets you. And, in any event,
that has zero to do with falsifiability. The results of any study stand
on their own unless and until they are demonstrated to be suspect, or
wrong. If the test is not designed to be falsifiable, it is a defective
design irrespective of how the data are analyzed or used. Perhaps you
need to brush up on what falsifiability means in test design.

<snip>

>>
>> Sorry, but you seem to be using a rather unique definition of "fact" as
>> "real scientists" make claims of fact for every such study.
>
> Complete nonsense. And you say this after bring up the null
> hypothesis. You might want to read up on the null hypothesis and what
> it proves and what it does not prove.
> http://en.wikipedia.org/wiki/Null_hypothesis

I suggest you follow your own recommendation.

>
>> The results
>> *are* facts, and are true, and applicable, within the constraints and
>> confidence interval of the test design. To believe otherwise would
>> require a refutation of statistics. If you doubt this, then please
>> explain exactly how many tests are required to result in "facts".
>
> No the results are not facts the results are data.

Data *are* objective facts. What do you think they are if not facts?

> Often in this kind
> of research one will find conflicting data. That is what no one who
> understands these kinds of things would ever draw a conclusion of fact
> from a single test. To say it would be a hasty conclusion would be an
> understatement.

Clearly you need to brush up on what constitutes "data", "facts", and
"conclusions". They are not interchangeable nor fungible. And you are
conflating "facts" with "conclusions". The only relevant conclusion I
saw in the subject post had to do with lack of data contravening known
physical and engineering principles, not citing any single test as
globally applicable.

Keith

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kaqq1...@news4.newsguy.com...

> But your "hyper-expensive" gear is not so "mind blowingly" better than the
> less
> expensive gear.

Misleading misappropriation of the word "your" noted.

> It is subtly different, usually marginally cleaner, especially in the
> top-end where the highs are less "grainy" and smoother than more
> run-of-the-
> mill components.

No reliable evidence of this seems to have been provided.

> But many people cannot (or will not) hear the differences.
> That's not their fault, really. Honest high-end equipment manufacturers
> use
> the best quality components in their audio gear. They use the best
> capacitors,
> the least noisy resistors, the finest potentiometers, the best grade of
> switches,
> etc.

There is no reliable evidence that any of this necessarily has any audible
benefits.

> This accounts for SOME of the high prices that these devices demand.

Costs with no benefits = waste.

> And it usually results in slightly better sound.

No reliable evidence of this seems to have been provided.

Repeating a false claim does not make it true.

> But as I have stated before, while these differences do exist,

Again no reliable evidence of this seems to have been provided and of course
repeating a false claim does not make it true.

> they are so small

Again no reliable evidence of this seems to have been provided and of course
it is still true repeating a false claim does not make it true.

[Scott]

And some of you clearly are not and clearly don't.

>
> > I would
> > think that if objectivists were genuinely interested in applying
> > science to the question of amplifier sound they would not move the
> > goal posts nor would they use ABX DBTs the way they have when it comes
> > to amplifier sound.
>
> The thread has nothing to do with "amplifier" sound.

Then take it up with the moderators. The subject has been brought up
so I addressed it.

>
> > That being typically breaking out ABX and failing
> > to ever control for same sound bias or even calibrate the sensitivity
> > of the test. Without such calibration a single null result tells us
> > very little about what was and was not learned about the sound of the
> > components under test.
>
> Careful reading would show I clearly stipulated such requirements need
> to be defined and accounted for.  Arguing in favor of my stated position
> isn't much of a refutation.

Careful reading *of the entire thread* would show that 1. Other people
besides you are involved. 2. Others have stipulated such requirements
are either unnecessary or don't exist at all. Just read the quoted
text in this post. It's there and because it's there it's relevant

>
>
>
> > But of course my point was the fact that no scientist worth his or her
> > salt would ever make dogmatic claims of fact based on the results of
> > any single ABX DBT null. And if one think that claims from
> > subjectivists should alter that fact then they simply don't understand
> > how real science deals with and interprets real scientific data.
>
> The "dogmatic" claims, as you describe them, were based on physics and
> engineering principles,

Really? Once again we have a bogus waving of the science flag. Do tell
us what "physics" stands behind the claim? And let me remind of just
what that claim was to begiin with. In this thread it was claimed On

Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote: "The
nice thing about testing for difference as ABX does is that if there
is no difference detected you know that the more expensive one is not

any better sounding." So please show us how this claim was based on
physics and engineering principles. In what part of physics is it
stated that one can draw hard conclusions from one null result done at
home? What engineering principle supports this claim?

> and the fact that listening tests, under
> controlled conditions, have not shown results that dispute those
> principles.

Please cite the principles you are refering to and the actual
listening tests. Hopefully for your sake you are not going to cite the
listening tests published in The Sensible Sound. ;-)

>  There was no claim, as I read it, that any individual test
> was applicable to all conditions.

You might want to read this again then. On Dec 14, 8:17 pm,

Barkingspyder <kmcke...@roadrunner.com> wrote: "The nice thing about
testing for difference as ABX does is that if there is no difference
detected you know that the more expensive one is not any better
sounding."

>  Quite the opposite in fact - where
> are the tests that contradict the the physics and engineering principles?
>

There you go waving the science flag again with nothing of substance
behind it. Please cite the physics and engineering principles you
believe support the claim that "The nice thing about testing for

difference as ABX does is that if there is no difference detected you

know that the more expensive one is not any better sounding." After
all, this is the specific claim I was challenging and others
apparently, including yourself, are defending.

> >>> Understanding he true significance of a single null result
> >>> does not require consideration of you or anyone else has been told by
> >>> other audiophiles.
>
> >> That would rest entirely upon how the null hypothesis is constructed,
> >> and may indeed include such claims.
>
> > No it does not. Real science builds it's conclusions on an
> > accumulation of research.
>
> No, every test has a conclusion, and is dispositive, if executed
> accurately, within the limitations of the specific test.

Within the limitations of the specific test. And within the
limitations of a home brewed ABX test one can not reasonable conclude
from a single null result that "if there is no difference detected you
know that the more expensive one is not any better sounding." That is
an erroneous and very unscientific conclusion.

>
> > Again if one understands how science works
> > they should know the real standing of one singular null result. That
> > being it is most certainly not something one can reasonably close the
> > books on and say that it is final proof of no difference.
>
> The "books" are clearly closed on that test group, under those test
> conditions.  To think otherwise is to deny the relevance of all tests

> under all conditions,

"that test group" being what? All tests being what? All conditions
being what? Your claim is way overly vague to even address.

>
>
>
> >>> For that to affect the weight placed on any single
> >>> test result would quite unscientific thinking.
>
> >> Again, simply not accurate with respect to the world of possible
> >> hypotheses.  Any null result for a discrimination test evaluating
> >> "obvious" differences will be significant, if not dispository, for that
> >> test and equipment, as long as the test is set up properly.
>
> > Sorry but you are plainly wrong. No scientist would ever put that much
> > stock in one test. It runs contrary to the very idea of
> > falsifiability, peer review or the idea of verification via repetition
> > of previous tests.very very unscientific
>
> Nonsense.

Nonsense to your claim of nonsense.

> Do one tox study and argue that 90% severe adverse effects
> doesn't mean anything.

Hold on here. You are putting words in my mouth. Where did I say the
test results of a single null "doesn't mean anything." Please quote
me. This is a typical straw man argument.

>  See how far that gets you.

It wouldn't get me v ery far but I know better than to do that. But
that is not what i am doing here.

>  And, in any event,
> that has zero to do with falsifiability.  The results of any study stand
> on their own unless and until they are demonstrated to be suspect, or
> wrong.  If the test is not designed to be falsifiable, it is a defective
> design irrespective of how the data are analyzed or used.  Perhaps you
> need to brush up on what falsifiability means in test design.

Perhaps you need to be reminded again of the original claim I was
disputing.

On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:

"The nice thing about testing for difference as ABX does is that if
there is no difference detected you know that the more expensive one
is not any better sounding."


>

> <snip>
>
>
>
> >> Sorry, but you seem to be using a rather unique definition of "fact" as
> >> "real scientists" make claims of fact for every such study.
>
> > Complete nonsense. And you say this after bring up the null
> > hypothesis. You might want to read up on the null hypothesis and what
> > it proves and what it does not prove.
> >http://en.wikipedia.org/wiki/Null_hypothesis
>
> I suggest you follow your own recommendation.

Oh I did. Here is what it says.
"The null hypothesis can never be proven. Data, such as the results of
an observation or experiment, can only reject or fail to reject a null
hypothesis"
Now what does that say about this claim? "The nice thing about

testing for difference as ABX does is that if there is no difference
detected you know that the more expensive one is not any better

sounding." Did you catch the word "KNOW" in there?

>
>
>
> >>   The results
> >> *are* facts, and are true, and applicable, within the constraints and
> >> confidence interval of the test design.  To believe otherwise would
> >> require a refutation of statistics.  If you doubt this, then please
> >> explain exactly how many tests are required to result in "facts".
>
> > No the results are not facts the results are data.
>
> Data *are* objective facts.  What do you think they are if not facts?

In the case of ABX DBTs they are merely results. The fact that any ABX
test can for any number of reasons wrought incorrect results makes it
pretty hard to call the results "facts" If they are facts then when
one ends up with conflicting data from different tests you have
conflicting "facts." do explain how that works.

>
> > Often in this kind
> > of research one will find conflicting data. That is what no one who
> > understands these kinds of things would ever draw a conclusion of fact
> > from a single test. To say it would be a hasty conclusion would be an
> > understatement.
>
> Clearly you need to brush up on what constitutes "data", "facts", and
> "conclusions".  They are not interchangeable nor fungible.

And yet you seem to be interchanging them. "Data *are* objective
facts." How ironic is that?

>  And you are
> conflating "facts" with "conclusions".

I am? Here is the conclusion I am challenging "The nice thing about

testing for difference as ABX does is that if there is no difference
detected you know that the more expensive one is not any better

sounding." His conclusion is a claim of fact. So who exactly is
conflating facts with conclusions?

> The only relevant conclusion I
> saw in the subject post had to do with lack of data contravening known
> physical and engineering principles, not citing any single test as
> globally applicable.

Cherry picking is also very unscientific. if that is the only
conclusion you saw in this thread then you missed the very conclusion
I have challenged in this thread. Just so you don't miss it again.

[nab...@hotmail.com]

On Wednesday, December 19, 2012 6:39:10 AM UTC-5, Scott wrote:

> Well, I am so glad you asked.

Not sure why. Let's take a look at the two key elements of the "data" you present.

You quote Howard Ferstler saying, "Even though a 68% correct score looks like there may have been significant audible differences with the 17 out of 25 mindnumbing trials I did, that score does achieve a 95% confidence level, indicating that the the choices were still attributable to chance."

You quote John Atkinson saying, "In other words, your own tests suggested you heard a difference..."

Howard is correctly interpreting the statistics here. John is not. A confidence interval is a hard target, not a rough idea you only have to get close to.

<snip>

> Howard Ferstler:
>
> " The data you are referring to was but a small part of the series.
> It was a fluke, because during the last part of that series of trials
> I was literally guessing. I just kept pushing the button and making
> wild stabs at what I thought I heard. After a while, I did not bother
> to listen at all. I just kept pressing the same choice over and
> over."
>
> IOW he was deliberately falsifying data in order to get a null result.
> I’d say that is proof positive of a same sound bias on the part of Mr.
> Ferstler wouldn’t you?

No, that's just what happens when you're doing a DBT and you really can't tell the difference. You have to guess. Howard's just being honest here. The only alternative is to abandon the test, but the outcome would be the same in both cases: No showing of audible difference.

> And this ABX DBT was published in The Sensible
> Sound despite the fact that the analysis was corrupted by a clear same
> sound bias but so was the data, deliberately!
> Ironically, due to an apparent malfunction in Tom Nousaine’s ABX box
> the attempt at spiking the results to get a null serendipitously
> wrought a false positive. So on top of that we have a mal functioning
> ABX box that Tom Nousiane has been using for all these ABX DBTs.

As explained above, there was no malfunction here. The only flaw is in Atkinson's interpretation of the results.

<snip>

> My goodness gracious talk about getting it all wrong. First ABX DBTs
> involves playback equipment. Pharaceutical trials do not so there is
> nothing to "calibrate" in pharmeceutical trials. BUT they do use
> control groups! That is in effect their calibration. without the
> control group the results mean nothing because there is no
> "calibrated" base to compare them to. So in effect they most
> defintiely are calibrated or they are tossed out as very very bad
> science and just plain junk. That is bias controlled testing 101.

That's not at all what calibration means, but just to humor you, let's pretend it is. In a DB drug trial, the intervention group needs to get a statistically better result than the control group. In an ABX test, the subjects need to get a statistically better result than chance. If the former is 'calibrated," then the latter is, too.

bob

[Arny Krueger]

"Scott" <S888...@aol.com> wrote in message

news:ajeb58...@mid.individual.net...

> On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote: "The
> nice thing about testing for difference as ABX does is that if there
> is no difference detected you know that the more expensive one is not
> any better sounding."

> So please show us how this claim was based on physics and engineering
> principles.

The above is not weird physics but weird rhetoric.

Obviously just because someone somewhere says something that mentions ABX,
that statement does not instantly and permanently binding on everybody who
pays attention to ABX tests.

It is like saying, "Since my buddy who is a Democrat thinks that (fill in
the blanks), President Obama is bound by that and must make it national
policy."

[Audio_Empire]

On Monday, December 17, 2012 6:49:26 AM UTC-8, Arny Krueger wrote:
> "Scott" <S888...@aol.com> wrote in message
>
> news:kai79...@news4.newsguy.com...
>
> On Dec 14, 8:21 pm, Audio_Empire <gmgrav...@comcast.net> wrote:
>
>
>
> > The person who was questioning to value of level matching did not seem
>
> > to be limiting his opinion to CDPs and amps.
>
>
>
> Seems like the backwards side of the argument. Doing comparisons of music
>
> players, DACs and amps without proper level matching seems to be the prelude
>
> to a massive waste of time. If the levels are not matched well enough then
>
> there will be audible differences, but we have no way of knowing that the
>
> causes are not our poor testing practices as opposed to any relevent
>
> property of the equipment being tested.

Also, the louder component will seem to the listening panel to be
"better" than the softer one. Just a dB or so difference is enough to
bias the panel toward the louder one.

> > You still have the same
>
> > problems in level matching that I stated above when dealing with
>
> > loudspeakers. In fact you have even more problems with radiation
>
> > pattern differences and room interfaces that make it even more
>
> > impossible to do a true level match.

In a speaker DBT, the one with more bass (as well as louder) will also
bias the listeners toward it.

[quoted text deleted -- deb]

[Audio_Empire]

Ah, but there is.... just not the kind that dyed-in-the-wool objectivists would be
willing to accept. That's why they are called objectivists. ;^)

[Scott]

On Dec 19, 9:41 am, nabo...@hotmail.com wrote:
> On Wednesday, December 19, 2012 6:39:10 AM UTC-5, Scott wrote:
> > Well, I am so glad you asked.
>
> Not sure why. Let's take a look at the two key elements of the "data" you present.
>
> You quote Howard Ferstler saying, "Even though a 68% correct score looks like there may have been significant audible differences with the 17 out of 25 mindnumbing trials I did, that score does achieve a 95% confidence level, indicating that the the choices were still attributable to chance."
>
> You quote John Atkinson saying, "In other words, your own tests suggested you heard a difference..."
>
> Howard is correctly interpreting the statistics here. John is not. A confidence interval is a hard target, not a rough idea you only have to get close to.

Um no, Howard interpreted the data backwards. he took 95% confidence
level to mean that it was a 95% likelihood that his results were due
to chance. The opposite is true. Atkinson was right. Ferstler was
wrong.

>
> <snip>
>
> > Howard Ferstler:
>
> >  " The data you are referring to was but a small part of the series.
> > It was a  fluke, because during the last part of that series of trials
> > I was literally guessing. I just kept pushing the button and making
> > wild stabs at what I thought I heard. After a while, I did not bother
> > to listen at all. I just kept  pressing the same choice over and
> > over."
>
> > IOW he was deliberately falsifying data in order to get a null result.
> > I’d say that is proof positive of a same sound bias on the part of Mr.
> > Ferstler wouldn’t you?
>
> No, that's just what happens when you're doing a DBT and you really can't tell the difference.

Nonsense. that si what happens when one tries to spike the data.
Sorry, there is no excuse on earth for someone to do what he did. he
says "I did not bother to listen at all I just kept pressing the same
choice over and over."." That is deliberate corruption of the data.
Done deal. If you can't see that for what it is we got nothin more to
talk about. really. It could not be more blatant.

> You have to guess.

He wasn't even guessing. He stopped listening. That is not doing an
ABX DBT properly. That is deliberately spiking data to get the desired
null.

> Howard's just being honest here.

Whoa hold here. he is being honest because he couldn't accept his own
result. Truth is his original article was plainly dishonest. If he
were being honest there he would have disclosed the fact that what he
was experiencing was exactly what he expected to experience
(expectation bias incarnate) and that he stopped listening and just
hit the same button. But he knew very well that this would make his
test worthless. But he'd rather admit his test was worthless than live
with the positive result. He just didn't understand the mistake in his
analysis or what the data was really saying so he went forward and
presented tests with deliberately spiked data as legitimate evidence
of amps sounding the same. Do you really think this is good science
much less honest journalism? if so let me fill you in. Any scientist
caught spiking data to gain a desired result is disgraced within the
scientific community.

> The only alternative is to abandon the test, but the outcome would be the same in both cases: No showing of audible difference.

How convenient. Circular logic incarnate.

>
> > And this ABX DBT was published in The Sensible
> > Sound despite the fact that the analysis was corrupted by a clear same
> > sound bias but so was the data, deliberately!
> > Ironically, due to an apparent malfunction in Tom Nousaine’s ABX box
> > the attempt at spiking the results to get a null serendipitously
> > wrought a false positive. So on top of that we have a mal functioning
> > ABX box that Tom Nousiane has been using for all these ABX DBTs.
>
> As explained above, there was no malfunction here. The only flaw is in Atkinson's interpretation of the results.

Seriously? You think an ABX machine that is giving a positive result
when you hit the same selection over and over again is not
malfunctioning? And again, Atkinson, a former science teacher gets the
analysis dead on. If you don't think so you are dead wrong end of
story.

>
> <snip>
>
> > My goodness gracious talk about getting it all wrong. First ABX DBTs
> > involves playback equipment. Pharaceutical trials do not so there is
> > nothing to "calibrate" in pharmeceutical trials. BUT they do use
> > control groups! That is in effect their calibration. without the
> > control group the results mean nothing because there is no
> > "calibrated" base to compare them to. So in effect they most
> > defintiely are calibrated or they are tossed out as very very bad
> > science and just plain junk. That is bias controlled testing 101.
>
> That's not at all what calibration means, but just to humor you, let's pretend it is. In a DB drug trial, the intervention group needs to get a statistically better result than the control group. In an ABX test, the subjects need to get a statistically better result than chance. If the former is 'calibrated," then the latter is, too.

Boy you are just getting this so wrong. Let me put this in the most
basic terms. Any such test needs negative and positive controls. what
are the negative controls in the ABX tests in either the Stereo review
tests or Howard Ferstler's ridiculous test? here is another question.
If two components sound different but the the testee *chooses* to

"not bother to listen at all. I just kept pressing the same choice

over and over." Are the results valid? Now let's see you navigate
these questions without using circular reasoning.

[KH]

On 12/19/2012 10:41 AM, nab...@hotmail.com wrote:
> On Wednesday, December 19, 2012 6:39:10 AM UTC-5, Scott wrote:
>

> <snip>
>
>> My goodness gracious talk about getting it all wrong. First ABX DBTs
>> involves playback equipment. Pharaceutical trials do not so there is
>> nothing to "calibrate" in pharmeceutical trials. BUT they do use
>> control groups! That is in effect their calibration. without the
>> control group the results mean nothing because there is no
>> "calibrated" base to compare them to. So in effect they most
>> defintiely are calibrated or they are tossed out as very very bad
>> science and just plain junk. That is bias controlled testing 101.

This is simply misinformed nonsense. The standard is control groups
where their use is ethical. Many, many drug trial are conducted without
control groups. You simply cannot ethically give a terminal patient a
placebo, nor a patient for whom lack of intervention results in
permanent injury. And yes, I *am* involved in one of these as we speak,
so please spare your speculation. In this regard, drug trials are
significantly different than many DBT's.

Keith

[KH]

On 12/19/2012 10:17 AM, Scott wrote:
> On Dec 19, 3:39 am, KH <keithahug...@q.com> wrote:
>> On 12/18/2012 10:18 AM, Scott wrote:
>>

>> Well, some of us *are* engaged in *real* science on a daily basis, and
>> do understand the precepts.
>
> And some of you clearly are not and clearly don't.

Yes, I am. Are you?

>>
>>> I would
>>> think that if objectivists were genuinely interested in applying
>>> science to the question of amplifier sound they would not move the
>>> goal posts nor would they use ABX DBTs the way they have when it comes
>>> to amplifier sound.
>>
>> The thread has nothing to do with "amplifier" sound.
>
> Then take it up with the moderators. The subject has been brought up
> so I addressed it.

Take it up with the thread TITLE.

>>
>>> That being typically breaking out ABX and failing
>>> to ever control for same sound bias or even calibrate the sensitivity
>>> of the test. Without such calibration a single null result tells us
>>> very little about what was and was not learned about the sound of the
>>> components under test.
>>
>> Careful reading would show I clearly stipulated such requirements need
>> to be defined and accounted for. Arguing in favor of my stated position
>> isn't much of a refutation.
>
> Careful reading *of the entire thread* would show that 1. Other people
> besides you are involved. 2. Others have stipulated such requirements
> are either unnecessary or don't exist at all. Just read the quoted
> text in this post. It's there and because it's there it's relevant

I didn't respond to the "entire" thread, but to a post. If you think
every post has to be responsive to the original post, you will likely be
disappointed.

Once again, I was responding a specific post and your response. That
would seem pretty obvious.

>> Quite the opposite in fact - where
>> are the tests that contradict the the physics and engineering principles?
>>
>
> There you go waving the science flag again with nothing of substance
> behind it. Please cite the physics and engineering principles you
> believe support the claim that "The nice thing about testing for
> difference as ABX does is that if there is no difference detected you
> know that the more expensive one is not any better sounding." After
> all, this is the specific claim I was challenging and others
> apparently, including yourself, are defending.

Again, you need to stay focused on the posts I was responding too if you
want to make sense of the discussion.

Gee, I thought "No scientist would ever put that much
stock in one test." was pretty clear. And please don't toss in the
dodge of "Barkingspyder said XXX", I clearly stated that "Any null

result for a discrimination test evaluating "obvious" differences will
be significant, if not dispository, for that test and equipment, as long

as the test is set up properly.", nothing more or less. Please explain
how that is nonsense in the context presented.

>
>> See how far that gets you.
>
> It wouldn't get me v ery far but I know better than to do that. But
> that is not what i am doing here.
>
>> And, in any event,
>> that has zero to do with falsifiability. The results of any study stand
>> on their own unless and until they are demonstrated to be suspect, or
>> wrong. If the test is not designed to be falsifiable, it is a defective
>> design irrespective of how the data are analyzed or used. Perhaps you
>> need to brush up on what falsifiability means in test design.
>
> Perhaps you need to be reminded again of the original claim I was
> disputing.

You were not, in the post I responded to, referring to the original post
(or were doing so in a manner to sufficiently cryptic to defy
identification), you were responding to Arny's post.

>
> On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:
> "The nice thing about testing for difference as ABX does is that if
> there is no difference detected you know that the more expensive one
> is not any better sounding."
>
>
>>
>> <snip>
>>
>>
>>
>>>> Sorry, but you seem to be using a rather unique definition of "fact" as
>>>> "real scientists" make claims of fact for every such study.
>>
>>> Complete nonsense. And you say this after bring up the null
>>> hypothesis. You might want to read up on the null hypothesis and what
>>> it proves and what it does not prove.
>>> http://en.wikipedia.org/wiki/Null_hypothesis
>>
>> I suggest you follow your own recommendation.
>
> Oh I did. Here is what it says.
> "The null hypothesis can never be proven. Data, such as the results of
> an observation or experiment, can only reject or fail to reject a null
> hypothesis"
> Now what does that say about this claim? "The nice thing about
> testing for difference as ABX does is that if there is no difference
> detected you know that the more expensive one is not any better
> sounding." Did you catch the word "KNOW" in there?

OK, so when the null hypothesis is "there are no significant differences
in sound between X and Y" it can be rejected when not true, right? The
qualifier - signficant, un-subtle, unmistakeable, etc. - defines the
sensitivity and precision required for the test (i.e. dismisses all of
the usual dodging and weaving about "forced choice stress", etc.). So,
given that the required sensitivity is trivial to achieve, we have the
following:

1. Physics and engineering principles, along with many years of
audiology and psychoacoustic experimentation provide an objective
threshold level below which differences are not detectable.
2. DBT tests of clearly sufficient sensitivity - given the claims - to
detect such differences if they exist, have all been negative.
3. There are no DBT data to contravene the expected results based on
engineering principles.
4. The null hypothesis is thus *accepted*, not disproven. This is the
basic mistake that most neophytes make. The null hypothesis is NOT that
two items/populations are different, it's that they are NOT different.
Thus one never needs to reject the null hypothesis to confirm
difference, just the opposite.

When one accepts the null hypothesis, one accepts that there is no
difference between the subjects/items/populations. So when you ask "Now
what does that say about this claim?", what is say is that one cannot
reject the null hypothesis that *they are the same*.

So no, accepting the null hypothesis doesn't "prove" there is no
difference. What is shows is that when clearly sensitive enough methods
are employed for evaluation, no differences are found. In this context,
where there are clear objective reasons why there *should* be no
differences, there is no additional burden on the proponents of the null
hypothesis.

<snip>

>> And you are
>> conflating "facts" with "conclusions".
>
> I am? Here is the conclusion I am challenging "The nice thing about
> testing for difference as ABX does is that if there is no difference
> detected you know that the more expensive one is not any better
> sounding." His conclusion is a claim of fact. So who exactly is
> conflating facts with conclusions?
>
>> The only relevant conclusion I
>> saw in the subject post had to do with lack of data contravening known
>> physical and engineering principles, not citing any single test as
>> globally applicable.
>
> Cherry picking is also very unscientific. if that is the only
> conclusion you saw in this thread

I wasn't responding to the entire thread - merely to you, and the post
you replied to. That should have been quite clear.

then you missed the very conclusion
> I have challenged in this thread. Just so you don't miss it again.
> "The nice thing about testing for difference as ABX does is that if
> there is no difference detected you know that the more expensive one
> is not any better sounding."

And this is, indeed, accurate in the situation where you have physical
or engineering based information that is corroborated by the ABX data.
ONLY in the presence of contravening data would this conclusion be
suspect. Where are those data?

Keith

[Scott]

On Dec 19, 12:38 pm, "Arny Krueger" <ar...@wowway.com> wrote:
> "Scott" <S888Wh...@aol.com> wrote in message
>
> news:ajeb58...@mid.individual.net...
>

> > On Dec 14, 8:17 pm, Barkingspyder <kmcke...@roadrunner.com> wrote:  "The
> > nice thing about testing for difference as ABX does is that if there
> > is no difference detected you know that the more expensive one is not
> > any better sounding."
> > So please show us how this claim was based on physics and engineering
> > principles.
>

> The above is not weird physics but weird rhetoric.

It's just what I said it was to begin with. Unscientific. Why you and
others have argued with me about saying that is kinda rd to
understand.

>
> Obviously just because someone somewhere says something that mentions ABX,
> that statement does not instantly and permanently binding on everybody who
> pays attention to ABX tests.

And who exactly have bound that statement to other than the person who
said it? You and others decided to argue with me when I called that
individual on this claim. Maybe you shouldn't have been so quick to
argue.

>
> It is like saying, "Since my buddy who is a Democrat thinks that (fill in
> the blanks), President Obama is bound by that and must make it national
> policy."

I suppose it is like that. But since I did not do that who cares?
kinda smells like straw burning don't it?

[nab...@hotmail.com]

On Wednesday, December 19, 2012 10:25:00 PM UTC-5, Scott wrote:
> On Dec 19, 9:41 am, nabo...@hotmail.com wrote:

> > You quote Howard Ferstler saying, "Even though a 68% correct score looks like there may have been significant audible differences with the 17 out of 25 mindnumbing trials I did, that score does achieve a 95% confidence level, indicating that the the choices were still attributable to chance."
>
> > You quote John Atkinson saying, "In other words, your own tests suggested you heard a difference..."
>
> > Howard is correctly interpreting the statistics here. John is not. A confidence interval is a hard target, not a rough idea you only have to get close to.

> Um no, Howard interpreted the data backwards. he took 95% confidence
> level to mean that it was a 95% likelihood that his results were due
> to chance. The opposite is true. Atkinson was right. Ferstler was
> wrong.

There is no point in carrying on a discussion about statistics who does not understand the most basic principles of statistics.

<snip>

> Seriously? You think an ABX machine that is giving a positive result
> when you hit the same selection over and over again is not
> malfunctioning?

He did not get a positive result. If you refuse to accept that, there is nothing more to say.

bob

[Scott]

This is a really old and tired debate. But I just want to clarify your
position on one thing before *I* close the books on this one. So it is
your position that Howard Ferstler is right when he says that his
results show a *95% confidence level that the results were due to
chance* and John Atkinson is wrong when he says the results show the
opposite, that they show a *95%, or more precisely a 94.6% confidence
level that the results were not due to chance?* Because *that is what
they actually claimed.* Just for the record are you really saying
Howard got that right and John got that wrong?

[ScottW]

On Dec 19, 7:25 pm, Scott <S888Wh...@aol.com> wrote:
> On Dec 19, 9:41 am, nabo...@hotmail.com wrote:
>
> > On Wednesday, December 19, 2012 6:39:10 AM UTC-5, Scott wrote:
> > > Well, I am so glad you asked.
>
> > Not sure why. Let's take a look at the two key elements of the "data" you present.
>
> > You quote Howard Ferstler saying, "Even though a 68% correct score looks like there may have been significant audible differences with the 17 out of 25 mindnumbing trials I did, that score does achieve a 95% confidence level, indicating that the the choices were still attributable to chance."
>
> > You quote John Atkinson saying, "In other words, your own tests suggested you heard a difference..."
>
> > Howard is correctly interpreting the statistics here. John is not. A confidence interval is a hard target, not a rough idea you only have to get close to.
>
> Um no, Howard interpreted the data backwards. he took 95% confidence
> level to mean that it was a 95% likelihood that his results were due
> to chance. The opposite is true. Atkinson was right. Ferstler was
> wrong.
>

The quote implies Howard took more than one test (each "test"
consisting of 25 trials). If he took 20 tests then it is quite
likely that 1 of the 20 will indicate a false positive result when
using 95% confidence as the conclusion.
It wouldn't be the first time Atkinson "cherry-picked" some data.
As far as just "punching a button"...I would agree the test requires
an honest effort
to discern a difference even if consciously...the subject does not
believe one exists.
That might disqualify some people as a useful subject for such a test.

A relatively simple way to control for this is to use multiple test
amps and to include in one test a control pair of amps that the
subject agrees do sound different. The subject should know this
pairing will occurr but not know when. Even if the subject decides in
a test that they cannot hear a difference in the amps and start
guessing does not invalidate the results. Failing to correctly
identify difference in the control pair would.

ScottW

[nab...@hotmail.com]

On Thursday, December 20, 2012 12:43:23 PM UTC-5, Scott wrote:

> This is a really old and tired debate. But I just want to clarify your
> position on one thing before *I* close the books on this one. So it is
> your position that Howard Ferstler is right when he says that his
> results show a *95% confidence level that the results were due to
> chance* and John Atkinson is wrong when he says the results show the
> opposite, that they show a *95%, or more precisely a 94.6% confidence
> level that the results were not due to chance?* Because *that is what
> they actually claimed.* Just for the record are you really saying
> Howard got that right and John got that wrong?

Neither is being precisely correct, but Howard at least got the conclusion right: His result did not achieve a 95% confidence level, and therefore he cannot reject the null hypothesis. John is, as they say, lying with statistics by trying to reset the confidence level after the fact. Had John said that there was a 94.6% probability that Howard's result was not due to chance, he would have been correct. To use the term "confidence level" in this context, and to further state that this "suggested" that Howard heard a difference, is an abuse of statistics. Your repeated claim that Howard got a positive result is similarly mistaken.

bob

[Scott]

On Dec 20, 9:43 am, ScottW <Scott...@hotmail.com> wrote:
> On Dec 19, 7:25 pm, Scott <S888Wh...@aol.com> wrote:
>
>
>
>
>
>
>
>
>
> > On Dec 19, 9:41 am, nabo...@hotmail.com wrote:
>
> > > On Wednesday, December 19, 2012 6:39:10 AM UTC-5, Scott wrote:
> > > > Well, I am so glad you asked.
>
> > > Not sure why. Let's take a look at the two key elements of the "data" you present.
>
> > > You quote Howard Ferstler saying, "Even though a 68% correct score looks like there may have been significant audible differences with the 17 out of 25 mindnumbing trials I did, that score does achieve a 95% confidence level, indicating that the the choices were still attributable to chance."
>
> > > You quote John Atkinson saying, "In other words, your own tests suggested you heard a difference..."
>
> > > Howard is correctly interpreting the statistics here. John is not. A confidence interval is a hard target, not a rough idea you only have to get close to.
>
> > Um no, Howard interpreted the data backwards. he took 95% confidence
> > level to mean that it was a 95% likelihood that his results were due
> > to chance. The opposite is true. Atkinson was right. Ferstler was
> > wrong.
>
> The quote implies Howard took more than one test (each "test"
> consisting of 25 trials).   If he took 20 tests then it is quite
> likely that 1 of the 20 will indicate a false positive result when
> using 95% confidence as the conclusion.
> It wouldn't be the first time Atkinson "cherry-picked" some data.
> As far as just "punching a button"...I would agree the test requires
> an honest effort
> to discern a difference even if consciously...the subject does not
> believe one exists.
> That might disqualify some people as a useful subject for such a test.

It does imply that. But it wasn't what happened. What was reported in
this particular case was actually one test with 25 trials.

>
> A relatively simple way to control for this is to use multiple test
> amps and to include in one test a control pair of amps that the
> subject agrees do sound different.  The subject should know this
> pairing will occurr but not know when.  Even if the subject decides in
> a test that they cannot hear a difference in the amps and start
> guessing does not invalidate the results.  Failing to correctly
> identify difference in the control pair would.
>

Scott, I completely agree that that is one very easy and reasonable
way to show an ABX test is in some way sensitive to differences. But
none of these ABX DBTs that are being touted as scientific proof on
the subject did that or anything like that even though all these tests
involved people with very strong opinions that all amps sound the
same. Everyone knew that it was ABX tests of amps. In the one Stereo
Review article about their big amplifier challenge one of the amps was
an old Futterman OTL. Even this amp fell into the sounds the same
category in the analysis of that data. If an underpowered antique OTL
isn't being heard as different that should tell you something about
that set of tests. Since then the objectivist camp has moved the goal
posts. Instead of claiming all amps sound the same they claim that all
amps sound the same or are not working properly to accommodate the
obvious fact that many tube amps most definitely sound different. So
what does the failure of the testees to hear an underpowered antique
Futterman OTL tell you about the sensitivity to differences of those
tests? And yet this old article is still being dragged out as
scientific proof.

And please note that Stereo review did review any number of tube amps
and preamps and claimed in every single case that they all sounded the
same as every other amp and preamp.

[nab...@hotmail.com]

On Thursday, December 20, 2012 12:43:32 PM UTC-5, ScottW wrote:

> It wouldn't be the first time Atkinson "cherry-picked" some data.

No, and as for criticizing the statistical reporting in other magazines, he should perhaps refrain from tossing stones out of his glass house. I recall one breathless report of a DBT proving an audible difference because the subjects "heard a difference fully half the time."

bob

[Arny Krueger]

"Scott" <S888...@aol.com> wrote in message

news:ajh9f3...@mid.individual.net...

> But none of these ABX DBTs that are being touted as scientific proof on

> the subject...

Since nobody who really understands science and statistics is claiming that
ABX DBTs are scientific proof of anything, you would appear to be arguing
with yourself.

It is fundamental to science that all of its findings are provisional until
better findings are obtained. Therefore the very concept of some kind of
final "scientific proof" is itself nonsense.

>In the one Stereo Review article about their big amplifier challenge one of

>the amps was an old Futterman OTL. Strike 1

The amp in question was not an old Futterman OTL but rather it was a modern
amplfiier (in new product production at or near the time of the tests) that
happened to pattern itself somewhat after the origional Futterman OTL tubed
amp. There were many differences. If memory serves it contained solid state
devices, perhaps some in the signal path.

> If an underpowered antique OTL isn't being heard as different that should
> tell you something about that set of tests.

The amp in question was not an antique and was never operated beyond its
realm of linear operation so it was not underpowered. Strike 2

Your lack of technical understanding of OTL tubed amplifiers seems to
include a lack of appreciation for the fact that an OTL amplifier removes
any output transformer from the signal path, thus removing a large source of
inherent nonlinear distoriton and bandwidth limits.

If there was any kind of a tubed amplifier that would be most likely to
sound like an equally transformerless SS amplifier, it might be one
patterned on the old Futterman design. Strike 3.

[nab...@hotmail.com]

On Thursday, December 20, 2012 4:16:46 PM UTC-5, Arny Krueger wrote:

> It is fundamental to science that all of its findings are provisional until
> better findings are obtained.

It's worth noting that my two DBT posts have drawn nearly 100 responses at this point, and yet not a single shred of "better findings" has been presented, despite assurances by at least two posters that such findings exist.

It's almost enough to make you doubt that any counter-evidence exists.

bob

[Audio_Empire]

I still believe that the only way to do a DBT/ABX of something as
subtly different as amplifiers should be done with each amp being
auditioned for as much as a half hour before switching to the other
amp. Use the same cuts from test CDs for each session, played in
the same order. Of course careful level matching and strict double-
blindness must still be maintained. I suspect that such a test might
uncover differences that short term, instantaneous switching doesn't
reveal.

[nab...@hotmail.com]

On Thursday, December 20, 2012 6:37:24 PM UTC-5, Audio_Empire wrote:

> I still believe that the only way to do a DBT/ABX of something as
> subtly different as amplifiers should be done with each amp being
> auditioned for as much as a half hour before switching to the other
> amp. Use the same cuts from test CDs for each session, played in
> the same order. Of course careful level matching and strict double

> blindness must still be maintained. I suspect that such a test might
> uncover differences that short term, instantaneous switching doesn't
> reveal.

As long as I've been reading RAHE (which is going on 15 years, I think), I've seen this belief expressed. One of these believers ought to try it sometime. Perhaps they will teach the world of psychoacoustics something. (I am not holding my breath.)

bob

[Audio_Empire]

On Thursday, December 20, 2012 1:16:46 PM UTC-8, Arny Krueger wrote:
> "Scott" <S888...@aol.com> wrote in message
>
> news:ajh9f3...@mid.individual.net...
>
>
>
> > But none of these ABX DBTs that are being touted as scientific proof on
>
> > the subject...
>
>
>
> Since nobody who really understands science and statistics is claiming that
>
> ABX DBTs are scientific proof of anything, you would appear to be arguing
>
> with yourself.
>
>
>
> It is fundamental to science that all of its findings are provisional until
>
> better findings are obtained. Therefore the very concept of some kind of
>
> final "scientific proof" is itself nonsense.
>
>
>
> >In the one Stereo Review article about their big amplifier challenge one of
>
> >the amps was an old Futterman OTL. Strike 1
>
>
>
> The amp in question was not an old Futterman OTL but rather it was a modern
>
> amplfiier (in new product production at or near the time of the tests) that
>
> happened to pattern itself somewhat after the origional Futterman OTL tubed
>
> amp. There were many differences. If memory serves it contained solid state
>
> devices, perhaps some in the signal path.

Tube amps are the exception. Many are designed to have the "tube sound" and
a DBT with a good solid-state amp will show definite differences that are by no
means subtle (in thet they stick out like a sore thumb. OTL amps are even more
so. Unless it uses a pair of transistors as the output stage, OTL have a relatively
high output impedance even if you parallel 8 pairs of output tubes! They just
can't be as neutral as a good S-S amp. The only time I've ever heard a OTL amp
sound great was when it was designed to be coupled to an electrostatic speaker.
Talk about a marriage made in heaven (or some-such place) the high output
impedance of the OTL and the high input impedance of the ESL, if designed to
be used together, eliminate two transformers.

>
>
> > If an underpowered antique OTL isn't being heard as different that should
>
> > tell you something about that set of tests.

Unless, as, I said above, the output stage is solid-state.

> The amp in question was not an antique and was never operated beyond its
>
> realm of linear operation so it was not underpowered. Strike 2

again it depends upon the OTL amp's output impedance. Futterman
did make several hybrid amps with tubed input and solid state
output. He called them Moscode amps. One was 150 Watts/channel and the
other was 300 Watts/Channel. They were called the Moscode 300 and the
Moscode 600 respectively. Was it one of those?

>
>
>
> Your lack of technical understanding of OTL tubed amplifiers seems to
>
> include a lack of appreciation for the fact that an OTL amplifier removes
>
> any output transformer from the signal path, thus removing a large source of
>
> inherent nonlinear distoriton and bandwidth limits.

...While introducing a fairly high output impedance unless you parallel a
dozen output tubes, and still you won't get the really low impedance
looking back from the speaker which is common with almost any
solid-state amp. Sometimes that's a good trade-off and sometimes it isn't.

> If there was any kind of a tubed amplifier that would be most likely to
>
> sound like an equally transformerless SS amplifier, it might be one
>
> patterned on the old Futterman design. Strike 3.

No, I don't think so. Most OTL amps don't have the speaker dampening characteristics
that SS amps have. IIRC, the Futterman design was an exception and had an output
impedance of something like 0.5 Ohms (don't take that to the bank, I may be mis-
remembering here. It's been a long time since I've auditioned a pair of them. The
only thing that I thought the OTL didn't do as well as a SS amp (or tube amp with output
transformers) is bass. The Futterman OTL amp gave really "wooly" bass.

[Audio_Empire]

I have tried it. And if there are any differences, one has a much better
chance of uncovering them if one really listens to the devices being
auditioned. You can't do that when two devices are being swapped
out for each other every few seconds (or even every couple of minutes).
As long as the auditions are truly double-blind, and the levels are carefully
matched to less than a dB, and the same varied demonstration material is
used in each instance, they are still true DBTs.

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kb0me...@news4.newsguy.com...

> On Thursday, December 20, 2012 1:16:46 PM UTC-8, Arny Krueger wrote:

>> "Scott" <S888...@aol.com> wrote in message

>> The amp in question was not an old Futterman OTL but rather it was a
>> modern
>> amplfiier (in new product production at or near the time of the tests)
>> that
>> happened to pattern itself somewhat after the origional Futterman OTL
>> tubed
>> amp. There were many differences. If memory serves it contained solid
>> state
>> devices, perhaps some in the signal path.

> Tube amps are the exception.

Contrary to popular belief, all tubed amps aren't the same. ;-)

> Many are designed to have the "tube sound" and
> a DBT with a good solid-state amp will show definite differences that are
> by no
> means subtle (in thet they stick out like a sore thumb. OTL amps are even
> more
> so. Unless it uses a pair of transistors as the output stage, OTL have a
> relatively
> high output impedance even if you parallel 8 pairs of output tubes! They
> just
> can't be as neutral as a good S-S amp. The only time I've ever heard a OTL
> amp
> sound great was when it was designed to be coupled to an electrostatic
> speaker.
> Talk about a marriage made in heaven (or some-such place) the high output
> impedance of the OTL and the high input impedance of the ESL, if designed
> to
> be used together, eliminate two transformers.

We now have proof of complete ignorance of the actual test conditions and
even the UUTs that were used a well-known ABX test that has been libeled in
this thread.

I see no appropriate reaction to that regrettable fact.

That kind of unrepentant ignorance raises serious doubts about any other
attempts at superior expertise or even basic credibility from the same
source.

The above is just baseless speculation presented as fact.

There is no reason for me to waste my time rebutting what appears to me to
be fantasy. I've got my facts straight. The people posting here from the
scientific viewpoint appear to have their facts straight.

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kb07f...@news3.newsguy.com...

> I still believe that the only way to do a DBT/ABX of something as
subtly different as amplifiers should be done with each amp being
auditioned for as much as a half hour before switching to the other
amp.

Been there, done that.

> Use the same cuts from test CDs for each session, played in
the same order.

Been there, done that.

> Of course careful level matching and strict double-
blindness must still be maintained.

Been there, done that.

> I suspect that such a test might
uncover differences that short term, instantaneous switching doesn't
reveal.

Didn't happen. So much for this round of hoops and sticks. ;-)

This appears to be a terribly unbalanced discussion. On the one side we seem
to have little but denial and speculation. On the other side we have over 35
years of hands-on experience with highly sophisticated real world testing of
dozens of amplifiers and equal numbers of DACs, signal processors and
players, some of it documented in the largest circulation consumer and
professional audio publications around.

[ScottW]

I agree. Selection of the source... A, B, or X should be under
subject control and for as long as the subject desires with whatever
material the subject desires before making a choice. My own choice
of material would not be limited to music. It masks rather than
illuminates differences.

ScottW

[nab...@hotmail.com]

On Friday, December 21, 2012 9:51:42 AM UTC-5, Audio_Empire wrote:
> On Thursday, December 20, 2012 6:46:30 PM UTC-8, nab...@hotmail.com wrote:
> > On Thursday, December 20, 2012 6:37:24 PM UTC-5, Audio_Empire wrote:


> > > I still believe that the only way to do a DBT/ABX of something as
> > > subtly different as amplifiers should be done with each amp being
> > > auditioned for as much as a half hour before switching to the other
> > > amp. Use the same cuts from test CDs for each session, played in
> > > the same order. Of course careful level matching and strict double
> > > blindness must still be maintained. I suspect that such a test might
> > > uncover differences that short term, instantaneous switching doesn't
> > > reveal.
> >
> > As long as I've been reading RAHE (which is going on 15 years, I think), I've seen this belief expressed. One of these believers ought to try it sometime. Perhaps they will teach the world of psychoacoustics something. (I am not holding my breath.)
>

> I have tried it.

No, you haven't, not with enough rigor to pass the laugh test. It's clear from your posts that you don't have a clear grasp of what is required to conduct a scientifically valid DBT.

>And if there are any differences, one has a much better
> chance of uncovering them if one really listens to the devices being
> auditioned. You can't do that when two devices are being swapped
> out for each other every few seconds (or even every couple of minutes).

There's solid science that says the opposite is true--comparing brief snippets of sound and switching quickly between them actually makes the test more sensitive to differences, not less. Sean Olive's speaker preference tests for Harman use per-speaker presentations measured in seconds, not minutes. If longer presentations worked better, it would be in Harman's economic interest to use them.

> As long as the auditions are truly double-blind, and the levels are carefully
> matched to less than a dB,

And here is clear evidence that you don't know how to do a DBT. A 1 dB difference is way too large. Try 0.1 dB. Also required is a forced-choice format and a statistically significant result over a meaningful number of trials.

bob

[Audio_Empire]

On Friday, December 21, 2012 10:38:52 AM UTC-8, nab...@hotmail.com wrote:
> On Friday, December 21, 2012 9:51:42 AM UTC-5, Audio_Empire wrote:
>
> > On Thursday, December 20, 2012 6:46:30 PM UTC-8, nab...@hotmail.com wrote:
>
> > > On Thursday, December 20, 2012 6:37:24 PM UTC-5, Audio_Empire wrote:
>
>
>
>
>
> > > > I still believe that the only way to do a DBT/ABX of something as
>
> > > > subtly different as amplifiers should be done with each amp being
>
> > > > auditioned for as much as a half hour before switching to the other
>
> > > > amp. Use the same cuts from test CDs for each session, played in
>
> > > > the same order. Of course careful level matching and strict double
>
> > > > blindness must still be maintained. I suspect that such a test might
>
> > > > uncover differences that short term, instantaneous switching doesn't
>
> > > > reveal.
>
> > >
>
> > > As long as I've been reading RAHE (which is going on 15 years, I think), I've seen this belief expressed. One of these believers ought to try it sometime. Perhaps they will teach the world of psychoacoustics something. (I am not holding my breath.)
>
> >
>
> > I have tried it.
>
>
>
> No, you haven't, not with enough rigor to pass the laugh test.

That's amusing. Anytime you strict "objectivists" hear of a result
that goes against your dogma, it's always "The test was not
conducted properly" Or "the test wasn't rigorous enough." I'll
say one thing. It certainly keeps you fellows safe and sound.

> It's clear from your posts that you don't have a clear grasp of what is required to conduct a scientifically valid DBT.

Well, then Bob. When you get finished laughing at me, why don't enlighten us
as to the correct methodology for conducting scientifically valid audio DBTs?

> >And if there are any differences, one has a much better
>
> > chance of uncovering them if one really listens to the devices being
>
> > auditioned. You can't do that when two devices are being swapped
>
> > out for each other every few seconds (or even every couple of minutes).
>
>
>
> There's solid science that says the opposite is true--comparing brief snippets of sound and switching quickly between them actually makes the test more sensitive to differences, not less. Sean Olive's speaker preference tests for Harman use per-speaker presentations measured in seconds, not minutes. If longer presentations worked better, it would be in Harman's economic interest to use them.

Here is just another reason why I think that DBTs for audio are a flawed
methodology. How, for instance, are you going to tell whether
one DAC resolves image specificity better than another with short snippets?
What if the "snippets" are studio produced with multi-miked and multi-
channel performances that HAVE NO image specificity? How is switching
between two DACs, for instance every couple of seconds going to show
the imaging characteristics of the DUTs, even if that info is present in the
program material? It can't. No wonder Scientifically valid DBTs so often
return a null result with the finding that everything sounds alike.
Thanks for confirming my doubts, Bob.

>
>
>
> > As long as the auditions are truly double-blind, and the levels are carefully
>
> > matched to less than a dB,
>
>
>
> And here is clear evidence that you don't know how to do a DBT. A 1 dB difference is way too large. Try 0.1 dB. Also required is a forced-choice format and a statistically significant result over a meaningful number of trials.
>
>
>
> bob

Would you mind telling me EXACTLY how anybody is going to match
levels to 1/10th of a dB? Even the most expensive audio equipment
doesn't have potentiometers on them that have that kind of resolution,
because audio doesn't need that kind of resolution. It can be done to less
than one dB easily enough, maybe even 1/2 of a dB is doable, but 1/10th?
That would require aerospace-spec 10-turn pots, and they're not exactly
cheap, or all that easy to source. And some here say that DBTs are easy
to set-up. Gimme a break.

[Audio_Empire]

I hadn't considered that. Yeah, I think a selection of drums or gun shots,
and other kinds of day-to-day sounds would be useful as part of the
mix of test material.

[Audio_Empire]

On Friday, December 21, 2012 7:26:02 AM UTC-8, Arny Krueger wrote:
> "Audio_Empire" <gmgr...@comcast.net> wrote in message
>
> news:kb07f...@news3.newsguy.com...
>
>
>
> > I still believe that the only way to do a DBT/ABX of something as
>
> subtly different as amplifiers should be done with each amp being
>
> auditioned for as much as a half hour before switching to the other
>
> amp.
>
>
>
> Been there, done that.
>
>
>
>
>
> > Use the same cuts from test CDs for each session, played in
>
> the same order.
>
>
>
> Been there, done that.
>
>
>
> > Of course careful level matching and strict double-
>
> blindness must still be maintained.
>
>
>
> Been there, done that.
>
>
>
>
>
> > I suspect that such a test might
>
> uncover differences that short term, instantaneous switching doesn't
>
> reveal.
>
>
>
> Didn't happen. So much for this round of hoops and sticks. ;-)

But isn't it possible that this is your inability or unwillingness to hear
these things that produces, for you, a sameness for everything you test
this way?

> This appears to be a terribly unbalanced discussion. On the one side we seem
>
> to have little but denial and speculation. On the other side we have over 35
>
> years of hands-on experience with highly sophisticated real world testing of
>
> dozens of amplifiers and equal numbers of DACs, signal processors and
>
> players, some of it documented in the largest circulation consumer and
>
> professional audio publications around.

And some of these Large circulation consumer professional audio publications
do note differences in sound. So, here we are at an impasse.

[nab...@hotmail.com]

On Friday, December 21, 2012 4:38:05 PM UTC-5, Audio_Empire wrote:

> That's amusing. Anytime you strict "objectivists" hear of a result
> that goes against your dogma, it's always "The test was not
> conducted properly" Or "the test wasn't rigorous enough." I'll
> say one thing. It certainly keeps you fellows safe and sound.>

Results? What results? I haven't seen any results. All I've seen is one unsupported claim by an anonymous Usenet poster who doesn't even know how to match levels.

> Here is just another reason why I think that DBTs for audio are a flawed
> methodology. How, for instance, are you going to tell whether
> one DAC resolves image specificity better than another with short snippets?

A DAC does not "resolve image specificity." A DAC puts out two separate channels of audio with virtually no crosstalk between them. These two channels do not meaningfully interact in any way until sound emerges from speakers. That's how an audio system works.

> What if the "snippets" are studio produced with multi-miked and multi-
> channel performances that HAVE NO image specificity? How is switching
> between two DACs, for instance every couple of seconds going to show
> the imaging characteristics of the DUTs, even if that info is present in the
> program material? It can't.

If that's what you think, you're free to do a DBT any way you like. But just do it (with at least one skeptical witness, please). Imagining what you think you'll hear doesn't count.

> Would you mind telling me EXACTLY how anybody is going to match
> levels to 1/10th of a dB? Even the most expensive audio equipment
> doesn't have potentiometers on them that have that kind of resolution,
> because audio doesn't need that kind of resolution. It can be done to less
> than one dB easily enough, maybe even 1/2 of a dB is doable, but 1/10th?
> That would require aerospace-spec 10-turn pots, and they're not exactly
> cheap, or all that easy to source. And some here say that DBTs are easy
> to set-up. Gimme a break.

A decent voltmeter should be all you need.

bob

[nab...@hotmail.com]

On Friday, December 21, 2012 4:38:45 PM UTC-5, Audio_Empire wrote:

> But isn't it possible that this is your inability or unwillingness to hear
> these things that produces, for you, a sameness for everything you test
> this way?

Sure it's possible for one test subject. But my original post lists numerous tests conducted by different groups with numerous subjects, all of which show the same general inability to hear differences among DACs.

Now, maybe everyone who's ever conducted a DBT for publication has stacked his test panel with skeptics. But if that's the case, it should have been easy for other, more "enlightened" publications to conduct their own DBTs, with their own true believers, and blow the existing science out of the water. Why haven't they done that?

I'll tell you why: Because they know as well as I do that the results would expose their reviewers as frauds and put them out of business.

bob

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kb2kt...@news6.newsguy.com...

> But isn't it possible that this is your inability or unwillingness to hear
> these things that produces, for you, a sameness for everything you test
> this way?

Didn't I answer this very same question the last time that the subject came
up?

The short answer is that we've done listening tests with dozens if not 100s
or 1,000s of people, many of which were specifically engaged because they
were outspoken advocates of the audibility of the issues being studied.

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kb2kr...@news6.newsguy.com...

On Friday, December 21, 2012 10:38:52 AM UTC-8, nab...@hotmail.com wrote:

> > There's solid science that says the opposite is true--comparing brief
> > snippets of sound and switching quickly between them actually makes the
> > test more sensitive to differences, not less. Sean Olive's speaker
> > preference tests for Harman use per-speaker presentations measured in
> > seconds, not minutes. If longer presentations worked better, it would be
> > in Harman's economic interest to use them.

> Here is just another reason why I think that DBTs for audio are a flawed
> methodology.

What is flawed is pretending that ABX is only done with short snippets when
the opposite has been published in places like Audio magazine and said many
times on this forum.

> How, for instance, are you going to tell whetherne DAC resolves image

> specificity better than another with short snippets?

There the thread goes downhill again, based on posts pretending that the
author is smarter than science.

Yawn.

I get very tired of correcting the same egregious and illogical mistakes,
every time the topic comes up.

[Audio_Empire]

On Friday, December 21, 2012 4:55:20 PM UTC-8, nab...@hotmail.com wrote:
> On Friday, December 21, 2012 4:38:45 PM UTC-5, Audio_Empire wrote:
>
>
>
> > But isn't it possible that this is your inability or unwillingness to hear
>
> > these things that produces, for you, a sameness for everything you test
>
> > this way?
>
>
>
> Sure it's possible for one test subject. But my original post lists numerous tests conducted by different groups with numerous subjects, all of which show the same general inability to hear differences among DACs.

I don't remember addressing you with that comment, but if you insist on
taking it personally, I have no real objection. OTOH, if those tests were
carried out in the way that proposed as "the right way", then I don't doubt
that all the subjects "show the same inability to hear differences among DACs."

> Now, maybe everyone who's ever conducted a DBT for publication has stacked his test panel with skeptics. But if that's the case, it should have been easy for other, more "enlightened" publications to conduct their own DBTs, with their own true believers, and blow the existing science out of the water. Why haven't they done that?

Can't say. What I can say is that most of audio writers who I know doubt,
as do I, the efficacy of current DBT testing methodologies for evaluating
audio components.

> I'll tell you why: Because they know as well as I do that the results would expose their reviewers as frauds and put them out of business.

You are more than welcome to hold fast to that belief. Personally, I think
the methodology is wrong. And you can quote chapter and verse of
Psychoacoustical dogma and statistical analysis 'till the end of time
and it won't change my mind. In fact I agree that using the type of DBT
that you and some others have been talking about, is almost surely
going to return a null result. By definition and design it HAS TO return
that result for a variety of reasons.

[Audio_Empire]

Like I told NAB, above. These kinds of tests are by their very design, incapable
of giving a positive result for a variety of reasons.

[nab...@hotmail.com]

On Saturday, December 22, 2012 10:29:56 AM UTC-5, Audio_Empire wrote:

> I don't remember addressing you with that comment, but if you insist on
> taking it personally, I have no real objection.

I wasn't taking it personally. I was just pointing out how wrong you were.

> Can't say. What I can say is that most of audio writers who I know doubt,
> as do I, the efficacy of current DBT testing methodologies for evaluating
> audio components.

And we all know why: Those tests expose them as the frauds they are.

> And you can quote chapter and verse of
> Psychoacoustical dogma and statistical analysis 'till the end of time
> and it won't change my mind.

Kinda sums it all up right there.

bob

[Audio_Empire]

Fine, then don't answer such threads. But it was NAB who INSISTED that DBTs are best served
by short snipits lasting only a few seconds. This debate was aimed at him, not you.

[Arny Krueger]

"Audio_Empire" <gmgr...@comcast.net> wrote in message

news:kb4jl...@news4.newsguy.com...

> What I can say is that most of audio writers who I know doubt,
> as do I, the efficacy of current DBT testing methodologies for evaluating
> audio components.

Based on the comments I've seen on RAHE the last 2 or so weeks, up-to-date
knowlege about the current state of DBT testing methodologies is uncommon.

The first big tipoff is all of the exclusive references to ABX, which in
2012 has nothing like an exclusive grip on the world of subjective testing.
In fact ABX was obsoleted for most of the kinds of testing that most audio
practitioners want to do on the day it was invented. And I invented it! ;-)
Seriously, though.

Here is what looks like a pretty good paper on the topic. Looks like it is
more than a decade old so no, it doesn't have the latest greatest wisdom,
but it looks pretty good compared to what I've seen on RAHE lately. Hope it
dispells the rapidly-gathering fog:

http://telos-systems.com/techtalk/00222.pdf