Fw: [SSE] “The Truth Wears Off”
Phillip Warren
Re: the scientific method (and `peer review') - this seems
interesting
(thanks to Billy Cox at Herald-Tribune Dec 20th)
http://www.biology.ualberta.ca/palmer/palmer.html
Richard
Palmer, biologist (University of Alberta) - "We cannot
escape the troubling
conclusion that some perhaps many cherished
generalities are at best
exaggerated in their biological
significance"
http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer
ANNALS
OF SCIENCE: THE TRUTH WEARS OFF - Is there something wrong
with the
scientific method?
by Jonah Lehrer
DECEMBER 13, 2010
Jonah Lehrer,
Annals of Science, “The Truth Wears Off,” The New
Yorker, December 13, 2010,
p. 52
Read the full text of this article at http://crayz.org/science.pdf
Scientific
Experiments; Decline Effect; Replicability;
Scientists; Statistics; Jonathan
Schooler; Scientific Theories
ABSTRACT: ANNALS OF SCIENCE about the
decline effect.
On September 18, 2007, a few dozen neuroscientists,
psychiatrists, and drug-company executives gathered in a hotel
conference room in Brussels to hear some startling news. It had
to do
with a class of drugs known as atypical or
second-generation antipsychotics,
which came on the market in the
early nineties. The therapeutic power of the
drugs appeared to be
steadily falling. A recent study showed an effect that
was less
than half of that documented in the first trials, in the early
nineties.
Before the effectiveness of a drug can be confirmed, it
must be
tested again and again. The test of replicability, as it’s known,
is the foundation of modern research. It’s a safeguard for the
creep of
subjectivity.
But now all sorts of well-established, multiply confirmed
findings have started to look increasingly uncertain. It’s as if
our
facts are losing their truth. This phenomenon doesn’t yet
have an official
name, but it’s occurring across a wide range of
fields, from psychology to
ecology. When Jonathan Schooler was a
graduate student at the University of
Washington, he discovered a
surprising phenomenon having to do with language
and memory that
he called verbal overshadowing. While Schooler was
publishing his
results in journals, he noticed that it was proving difficult
to
replicate his earlier findings.
Mentions psychologist Joseph Banks
Rhine, who conducted several
experiments dealing with E.S.P.
In 2004,
Schooler embarked on an imitation of Rhine’s research
in an attempt to test
the decline effect.
The most likely explanation for the decline is an obvious
one:
regression to the mean. Yet the effect’s ubiquity seems to
violate
the laws of statistics. Describes Anders Møller’s
discovery of the theory of
fluctuating asymmetry in sexual
selection.
Mentions Leigh Simmons and
Theodore Sterling.
Biologist Michael Jennions argues that the decline
effect is
largely a product of publication bias.
Biologist Richard
Palmer suspects that an equally significant
issue is the selective reporting
of results—that is, the subtle
omissions and unconscious misperceptions, as
researchers struggle
to make sense of their results.
Mentions John
Ioannidis.
In the late nineteen-nineties, neuroscientist John Crabbe
investigated the impact of unknown chance events on the test of
replicability. The disturbing implication of his study is that a
lot of
extraordinary scientific data is nothing but noise.
This suggests that
the decline effect is actually a decline of
illusion. Many scientific
theories continue to be considered true
even after failing numerous
experimental tests. The decline
effect is troubling because it reminds us
how difficult it is to
prove anything.
---
Ray D
