Noii lombrosieni
Akatyst
Social neuroscientists criticized for exaggerating links between brain
activity and emotions.
Alison Abbott
A study attacking some of the most prominent research in the
burgeoning field of social neuroscience is flawed and unfair,
according to top scientists who have been accused of overselling their
results.
Brain imaging is used to assess neural mechanisms in social
behaviour.Brain imaging is used to assess neural mechanisms in social
behaviour.SocialBrainLab, Univ. Med. Center Groningen
Social neuroscience is the study of the neurobiological mechanisms
underlying social behaviour. The field frequently uses functional
magnetic resonance imaging (fMRI) to reveal which brain areas are
activated while a subject is exposed to specific social interactions —
for example, situations that may evoke jealousy or the perception of
unfairness.
But a no-holds-barred paper1, accepted for publication in Perspectives
on Psychological Science and already circulating widely on the
Internet, claims that many studies in the field are worthless because
brain imaging data have been poorly analysed.
The paper was written by Edward Vul, a PhD student supervised by
neuroscientist Nancy Kanwisher at the Massachusetts Institute of
Technology in Cambridge, along with psychologists at the University of
California, San Diego, including Harold Pashler.
The paper has touched a nerve: brain imaging studies were derided by
some as "the new phrenology" when they became common 15 years ago, and
interpretations of their highly complex data were denounced as naive.
But those directly attacked say they are familiar with, and avoid, the
pitfalls.
Vul and his co-authors say they wrote the paper because they were
concerned by what they considered to be the "implausibly high
correlations" reported between brain activation and particular forms
of behaviour, and the lack of methodological details provided. So they
selected 54 papers in social neuroscience and sent a brief
questionnaire to the authors requesting details of their analyses.
They concluded that in a 'red list' of 31 cases — often in high-
profile journals, including Nature and Science — the authors made
fundamental errors in data handling and statistics.
They particularly criticize a 'non-independence error', in which bias
is introduced by selecting data using a first statistical test and
then applying a second non-independent statistical test to those data.
This error, they say, arises from selecting small volumes of the
brain, called voxels, on the basis of their high correlation with a
psychological response, and then going on to report the magnitude of
that correlation. "At present, all studies performed using these
methods have large question marks over them," they write.
In a rebuttal2, four authors of different red-list papers explain that
there was no non-independence error because calculating the size of
the correlation is not a statistical test.
Appropriate corrections ensure that the correlations between the
selected voxels and psychological responses are likely to be real, and
not noise, they add. And the strictness of the correction means that
those correlations are necessarily less frequent and of higher
magnitude — a situation far from implausible.
Vul and his colleagues also claim that the magnitude of correlation is
limited by the average reliability of fMRI data generally. Not so, the
accused respond: although the reliability of fMRI is very variable,
the upper ceiling on data quality is extremely high.
The swift rebuttal was prompted by scientists' alarm at the speed with
which the accusations have spread through the community. The
provocative title — 'Voodoo correlations in social neuroscience' — and
iconoclastic tone have attracted coverage on many blogs, including
that of Newsweek. Those attacked say they have not had the chance to
argue their case in the normal academic channels.
"I first heard about this when I got a call from a journalist,"
comments neuroscientist Tania Singer of the University of Zurich,
Switzerland, whose papers on empathy are listed as examples of bad
analytical practice. "I was shocked — this is not the way that
scientific discourse should take place." Singer says she asked for a
discussion with the authors when she received the questionnaire, to
clarify the type of information needed, but got no reply.
"We didn't disclose all our potential criticisms before asking these
people to tell us things that should have already been in their method
sections," says an unrepentant Vul. "Would they have described their
methods differently if we had?"
They would indeed, says biopsychologist Turhan Canli of Stony Brook
University, New York, who has four papers on the red list. He argues
that the questionnaire, which was billed as taking no more than a few
minutes to complete, was not able to capture the rationale for using
particular analyses.
ADVERTISEMENT
Nature Methods
The article is scheduled for publication in September, alongside one
or more replies. But the accused scientists are concerned that the
impression now being established through media reports will be hard to
shake after the nine-month delay. "We are not worried about our close
colleagues, who will understand the arguments. We are worried that the
whole enterprise of social neuroscience falls into disrepute," says
neuroscientist Chris Frith of University College London, whose Nature
paper3 on response to perceived fairness was called into question.
"On the other hand, we all agree that there is a kernel of truth in
what Vul and his colleagues write about some of the literature being
shaky," adds Christian Keysers of the University of Groningen in the
Netherlands, whose 2007 paper in NeuroImage on empathy was
highlighted4. "We can never be reminded often enough of the importance
of good statistical practice."
*
References
1. Vul, E. et al. www.pashler.com/Articles/Vul_etal_2008inpress.pdf
2. Jabbi, M. et al. www.bcn-nic.nl/replyVul.pdf
3. Singer, T. et al. Nature 439, 466–469 (2006). | Article |
PubMed | ISI | ChemPort |
4. Jabbi, M. et al. NeuroImage 34, 1744–1753 (2007). |
Article | PubMed |
Comments
Reader comments are usually moderated after posting. If you find
something offensive or inappropriate, you can speed this process by
clicking 'Report this comment' (or, if that doesn't work for you,
email rede...@nature.com). For more controversial topics, we reserve
the right to moderate before comments are published.
*
"I was shocked ? this is not the way that scientific discourse
should take place." It is just my personal belief that this is indeed
the way scientific discourse needs to take place every so often. I
respect Vul for making such direct criticisms of imaging methods. From
experience working in an fMRI lab - there are students and faculty who
are very knowledgeable and have a really strong grasp of their
methodology, from both the data acquisition end [experimental design
(hybrid, ER, etc.), pulse sequencing, explain their use of certain
image imaging parameters (slice thickness, etc.)] to the data analysis
[can explain what each data manipulation is doing to the data (things
like 3dDeconvolve, unwarping, fieldmaps, motion correction, etc.)].
And there are others who just go down the list of plug-ins like a
recipe, and if it gets an area of the brain to light-up, great, they
will explain it in retrospect. The amount of published research from
imaging experiments has drastically increased over the last 10 years,
and if Vul can put it into the conscience of an experimenter that they
better be sure they can explain themselves, then we will all benefit,
particularly those who don't work in the field of brain imaging, and
are at the mercy of the reviewers to assure them that those images
weren't made in Photoshop.
o 14 Jan, 2009
o Posted by: Phoenix Monakhos
*
How are mirror neurons incorporated/prevented into conscious
behavior in a social context. How do the emotions affect mirror
neurons? To what degree is the effect emotion driven? Under what
circumstances? Peter Templar
o 14 Jan, 2009
o Posted by: Laurie Simon
*
The critcised authors complain about the immediate publicity of
the criticisms. This may indeed not be the way that scientific
discourse should normally take place. However, given the
sensationalism with which some of the questioned findings were
publicised, it seems appropriate that the criticisms address the same
audience. Journals that accept manuscripts according to the chance
that they make headlines in the popular press may want to reconsider
this strategy.
o 15 Jan, 2009
o Posted by: Fritz Funke
*
For those interested, you can find our response to this reply
here: http://edvul.com/voodoorebuttal.php Cheers, Ed.
o 15 Jan, 2009
o Posted by: Ed Vul
*
RE: Self-criticism and Critics in Science -- As a retired (and
published) scientist and now an independent reader-writer in the
progress and philosophy of neuroscience, I think it is encumbered upon
all competent scientists to be self-critical in their own work; and
that all their published work (despite the normal peer-review
mechanism) shall be subject to the open scrutiny (and test of
reproducibility of their published work) by other competent scientists
as well as interested readers-critics alike. I thought Vul et al had
acted properly and professionally, and that their own published work
shall now be subject to the scrutiny of other competent
neuroscientists as well. As an ongoing competitive human endeavor, all
published scientific work (including those of social science,
psychology, etc) shall withstand or fall by the test of their own
weight and self-criticism as well as the criticisms of others,
including those of non-scientists and philosophers alike. Best wishes,
Mong 1/15/9usct1:45p; author "Decoding Scientism" and "Consciousness &
the Subconscious" (works in progress since July 2007), "Gods, Genes,
Conscience" (2006: http://www.iuniverse.com/bookstore/book_detail.asp?isbn=0595379907
) and "Gods, Genes, Conscience: Global Dialogues Now" (blogging avidly
since 2006: http://www2.blogger.com/profile/18303146609950569778 ).
o 15 Jan, 2009
o Posted by: Mong H Tan, PhD
Poliot
"Akatyst" <lind...@aol.com> wrote in message
news:fae28b03-48dd-43a4...@n33g2000pri.googlegroups.com...
Brain imaging studies under fire - Nature
Social neuroscientists criticized for exaggerating links between brain
activity and emotions.
Alison Abbott
cum adica "voi" lombrosienii, mai prikoki?