New autism study: When smart scientists make stupid mistakes

36 views
Skip to first unread message

news.omega

unread,
Nov 26, 2007, 6:18:04 PM11/26/07
to freepa...@googlegroups.com
Dear Dr Mutter,
Here is a section from Dr Hitlan's website and below Dan Olmstedt's
article "When smart scinetists make stupid mistakes".

Kind regards,

DK

http://www.uni.edu/~hitlan/ <http://www.uni.edu/%7Ehitlan/>

DeSoto, M. C. & *Hitlan, R. T.* (2007). Blood levels of mercury are
related to diagnosis of Autism: A reanalysis of an important data set.
/Journal of Child Neurology, 22, 1321-1323./
------------------------------------------------------------------------
The question of what is leading to the apparent increase in^ autism is
of great importance. Like the link between aspirin^ and heart attack,
even a small effect can have major health^ implications. If there is any
link between autism and mercury,^ it is absolutely crucial that the
first reports of the question^ are not falsely stating that no link
occurs. We have reanalyzed^ the data set originally reported by Ip et
al. in 2004 and have^ found that the original /p/ value was in error and
that a significant^ relation does exist between the blood levels of
mercury and^ diagnosis of an autism spectrum disorder. Moreover, the
hair^ sample analysis results offer some support for the idea that^
persons with autism may be less efficient and more variable^ at
eliminating mercury from the blood.^

^See also: http://www.ageofautism.com/2007/11/when-smart-scie.html

------------------------------------------------------------------------

* Hitlan, R. T.* (2007, August 1). Implicit Attitude Measures:
Advances and Controversy [Review of the book - Implicit Measures of
Attitudes]. /PsycCritiques—Contemporary Psychology: APA Review of Books/.* *

Walsh, B. M., & *Hitlan, R. T.* (2007). Organizational stress:
Investigating the impact of duel harassment experiences on appraisal and
outcomes. /North American Journal of Psychology, 9,/ 331-346.

* * *Hitlan, R. T.,* Carrillo, K., Aikman, S. N., & Zarate, M. A.
(2007). Attitudes toward immigrant groups and the effects of the 9/11
terrorist attacks. Peace and Conflict: Journal of Peace Psychology, 13,
1-18. * ** *

* * * Hitlan, R. T*., Schneider, K. T., & Walsh, B. (2006).
Upsetting behavior: Reactions to personal and bystander sexual
harassment experiences. /Sex Roles: A Journal of Research/, /55, 187-195./

* * * Hitlan, R. T.* (2006, November 8). Workplace Violence and
Aggression: The Dark Side of Organizations [Review of the book -
Handbook of Workplace Violence]. /PsycCritiques—Contemporary Psychology:
APA Review of Books, 51 (No. 45), Article 9/.* *

* * *Hitlan, R. T.* & DeSoto, M. C.(2006, March 8). The pros and
cons of growing up in the electronic age [Review of the book
Imagination and Play in the Electronic Age].
/PsycCritiques—Contemporary Psychology: APA Review of Books/, 51 (No.
10), Article 17.

* Hitlan, R. T.*, Cliffton, R. J., & DeSoto, M. C. (2006). Perceived
Exclusion in the Workplace: The Moderating Effects of Gender on
Work-Related Attitudes and Psychological Health. /North American Journal
of Psychology, 8,/ 217-236.

* Hitlan, R. T*., Kelly, K., Schepman, S., Schneider, K. T., &
Zarate, M. A. (2006). Language Exclusion and the Consequences of
Perceived Ostracism in the Workplace . /Group Dynamics: Theory,
Research, and Practice, 10/, 56-70/. /

* * *Hitlan, R. T*. (2006, in press). The Ringelmann Effect.
/Encyclopedia of Social Psychology. /Sage Publications.

DeSoto, M. C., & *Hitlan, R. T.* (2005, October 26). Transgender,
transsexual, or intersexual: A plight by any other name would tell as
tragic [Review of the book The Riddle of Gender: Science Activism, and
Transgender Rights. /PsycCritiques—Contemporary Psychology: APA Review
of Books, 40/ (No. 43), Article 8.

Zárate, M. A., Garcia, B, Garza, A. A., & *Hitlan, R. T. *(2004).
Cultural threat and perceived realistic group conflict as predictors of
attitudes towards Mexican immigrants. / Journal of Experimental Social
Psychology, 40, 99-105. /

Schneider, K. T., & *Hitlan, R. T.*, & Radhakrishnan, P. (2000) An
examination of the nature and correlates of ethnic harassment
experiences in multiple contexts. /Journal of Applied Psychology,
85(1/), 3-12.

Estrada, A. X., Berggren, A. W., *Hitlan, R. T.*, & Schneider, K.
T. (1999). Sexual harassment of female officers and cadets in the
Swedish Armed Forces. In M. Dansby (Ed.), Proceedings of the Defense
Equal Opportunity Management Institute,3. DEOMI: Cocoa Beach, FL.

/*Manuscripts Submitted for Publication:*/

* Hitlan, R. T.*, Kelly, K, & Zárate, M. A. (under review). Using
language to exclude: The effects of language-based exclusion on mood and
expressed prejudice. Manuscript submitted for publication.

* Hitlan, R. T.*, Noel, J. M., Kelly, K., & Zárate, M. A. (under
review). The Nature and Correlates of Workplace Exclusion. Manuscript
submitted for publication.

*Hitlan, R. T.*, & Noel, J. (under review). The Influence of
Workplace Exclusion and Personality on Deviant Workplace Behaviors: An
Interactionist Perspective. Manuscript submitted for publication.

*/Conference Presentations:/*

*Hitlan, R. T.*, Schneider, K.T., & Walsh, B. (May, 2006). The
impact of personal and bystander harassment experiences on appraisal.
poster accepted for presentation at the annual meeting of the Midwestern
Psychological Association conference, Chicago, IL

B.M. Walsh & *Hitlan, R. T.* (March, 2006). The Impact of Sexual
and Bystander Harassment on Female Employees' Physical Health. Poster
presented at the annual meeting of the Iowa Psychological Association.

* Hitlan, R. T.*, & Schneider, K. T. (January, 2006). The impact of
multiple harassment stressors on psychological distress using the
SCL-90. Poster pesented at the annual meeting of the Society for
Personality and Social Psychology conference, Palm Springs, CA.

*Hitlan, R. T.*, & Harden, J. (2004, April). The impact of
workplace exclusion and personality on workplace attitudes and
behaviors. In K. T. Schneider (chair), Emerging workplace diversity
issues: Ethnicity, bilingualism, and workplace exclusion. Symposium
paper presented at the Society for Industrial and Organizational
Psychology, Chicago, IL. abstract
<http://www.uni.edu/%7Ehitlan/KSchneider_SympSiop04.pdf>

Harden, J. & *Hitlan, R. T.* (2004, January). Predicting
antisocial workplace behaviors: Examining the interaction between social
exclusion and personality. Poster presented at the Society for
Personality and Social Psychology (SPSP), Austin, TX. abstract
<http://www.uni.edu/%7Ehitlan/spsp2004poster.pdf>

*Hitlan, R. T.*, & Harden, J. (2004, January). Exclusion at work:
Examining the relation between exclusion and work-related attitudes and
behaviors. In R. T. Hitlan & K. M. Kelly (Chairs), Broadening our
perspective: New directions in the study of social exclusion and
acceptance. Symposium paper presented at the Society for Personality and
Social Psychology (SPSP), Austin, TX. abstract
<http://www.uni.edu/%7Ehitlan/spsp2004.pdf>

*Hitlan, R. T.*, Zárate, M. A. & Schneider, K.T. (2003, April).
Language exclusion: Its effects on psychological need-threat, workplace
attitudes, and intergroup relations. In Bergman M. (Chair), Multiple
identities and discrimination: Examining intersections of
characteristics. Symposium presented at the Society for Industrial and
Organizational Psychology (SIOP), Orlando, Fl. slides. PDF
<http://www.uni.edu/%7Ehitlan/SIOP03.pdf> (© do not copy or cite without
permission)

*Hitlan, R. T.*, Kelly, K. & Zárate, M. A. (2003, February). The
Effects of Language-based Exclusion on Mood, and Psychological Threat.
Poster presented at the annual meeting of the Society for Personality
and Social Psychology conference, Los Angeles, CA.

Zárate, M.A., *Hitlan, R. T.*, Garza, A., & Aikman, S. (2002,
October). Attitudes toward Mexican and Arab immigrants as a function of
the September terrorist attacks. Paper presented at the annual meeting
of the Society for Experimental Social Psychology, Columbus, OH.
slides.PDF <http://www.uni.edu/%7Ehitlan/zaratearab.pdf> (© do not copy
or cite without permission)

http://www.ageofautism.com/2007/11/when-smart-scie.html


11/19/2007


WHEN SMART SCIENTISTS MAKE STUPID MISTAKES

Bullshit By Mark Blaxill, Editor-at-Large

In the Orwellian language of industry-funded research, the world has
only two kinds of science: "sound science" and "junk science." So
whenever you hear someone attack a piece of work as "junk science" it's
time to put your bullshit filters on. Like a political campaign,
financially-involved partisans in a scientific controversy have learned
that negative campaigning is an effective tactic.

Instead of dealing with facts, analysis and data, the tried and true
partisan approach is to launch an ad hominem attack on the character,
motivations, the sanity or even the home décor of the targeted analyst.
Facts and data are awfully boring after all and most journalists have
been conditioned not to actually read the science they cover. Instead,
they kowtow to "experts" who filter research for them. And if a
fancy-sounding expert denounces troubling work as "junk" or is willing
to don his white coat and intone for the camera that "science has
spoken" on a controversial topic, well then that's how it is, right?

Wrong. The scientific process has little to do with class distinctions
between respectable gentlemen and dangerous junkmeisters. It's far less
socially aware: it's about evidence. And evidence is a funny thing. It's
democratic and unpredictable. It has a way of not cooperating with
comfortable orthodoxies and appealing theories. And when problems are
complex, the evidence has a funny way of hiding behind bad design,
methodological errors and investigator bias. In our world of new
environmental diseases and, of course, an epidemic of chronic disease,
the great scientific issue of our time is not the struggle between "junk
science" and respectable science, it's the simple question, why are so
many children sick?

And in seeking the answer to that question, there is really only one
enemy. Bad analysis.

Bad analysis can happen many ways. Biased study designs can lead to bias
in the evidence base. Useful data can be processed poorly or with errors
in interpretation. Important insights can be lost or suppressed in favor
of the preferred insights. Or perfectly good evidence and data can be
misrepresented. In short, smart scientists (because most of the people
who submit scientific papers for publication are undeniably smart) make
mistakes. Sometimes these mistakes are innocent. Sometimes they're a
result of someone stretching their findings to defend their financial or
ideological interest.

But sometimes smart people just get things wrong. And because the peer
review process is an imperfect filter, sometimes mistakes get into print
that are just plain stupid.

In this month's Journal of Child Neurology, there's a fascinating new
report on autism and mercury that exposes a really large and really
stupid mistake. Two psychologists from the University of Northern Iowa
Catherine DeSoto and Robert Hitlan examined a paper published in 2004 in
the same journal by a group from the University of Hong Kong looking at
mercury exposure in children. The Hong Kong study was led by Patrick Ip
and looked at mercury levels in blood and hair for 7 year olds with
autism and compared the results to normal controls. In their paper, Ip
et al reported findings that went against the "poor excretor" hypothesis
of autism and mercury, claiming that there was no difference in any
measure of mercury exposure and excretion between their autism children
and control children. The last line of their abstract (the only part of
the study most scientists ever read) was surprisingly categorical: "the
results from our cohort study…indicate that there is no causal
relationship between mercury as an environmental neurotoxin and autism."

DeSoto and Hitlan took a preliminary look at Ip's analysis and were
concerned. "While attempting to estimate the effect size based on the Ip
et al statistics, we realized that the numbers reported by Ip et al
could not be correct." So they went back and looked at the data again.
What they concluded, looking at exactly the same data, was that a better
analysis led to the opposite conclusion. [Full disclosure: I've neither
met nor corresponded with DeSoto and Hitlan, but they described a study
on which I was a co-author as the "most direct test of the hypothesis
that autistic children may be deficient in terms of the ability to
remove mercury from circulation." I agree with them.]

I recommend that you read DeSoto and Hitlan's paper. It's quite short
and, despite a necessary discussion of the statistical details, quite
accessibly written. It's also wise. In summary, their analysis revealed
the key flaws in the Hong Kong team's study design as well as a number
of important effects that Ip et al overlooked.

The Ip et al study never attracted much attention in the first place
because their result was unimportant. They tested blood and hair mercury
levels in 7 year olds. No one has ever claimed that mercury levels in 7
year olds would have anything to do with the kinds of mercury exposures
required to induce autism in infant brains. More than anything else, the
mere publication of this article demonstrated the kind of sloppy logic
that pervades the autism debate. All that Ip et al demonstrated was that
7 year old Chinese children had relatively high levels of mercury in
their hair. Their strongly worded conclusion that there was no link
between mercury and autism was overly aggressive and, frankly, just
plain silly.

But DeSoto and Hitlan, their attention attracted by a clear statistical
error, started asking some smart questions. First, and most
surprisingly, they used the basic statistics reported by Ip et al to
calculate that there was a significant elevation in blood mercury levels
(higher by more than 10%) in the Chinese 7 year old autism group. [What
this means is anyone's guess. Perhaps these older children remain closer
to the environmental sources of mercury that placed them at greater risk
of autism to begin with.] Second, and more importantly, they deduced
that the differences between the hair and mercury excretion patterns in
the sample also supported the idea that the children in the autistic
group were poor excretors. Using the expected observation that the blood
and mercury levels were highly correlated in the sample (blood mercury
levels explained 75% of the differences in hair mercury) they discovered
that "the relationship between blood levels of mercury and mercury
excreted in the hair is reduced for those with autism compared to
non-autistic persons; furthermore, the difference between autistic
persons and nonautistic persons is most pronounced at high levels of
mercury."

This is an important and unexpected finding. It supports one of the
central hypotheses at the heart of the autism-mercury controversy and
suggests that the excretion deficit in autistic children might persist
longer than anyone had guessed. Why did Patrick Ip and his team miss
this point? Well first of all, they made a simple calculation error; an
error that their biases led them to believe was a valid result. Second,
they had a failure of imagination. It simply never occurred to them,
that their data contained evidence supporting the hypothesis they were
so eager to refute. If it weren't for the sharp eyes and diligent work
of DeSoto and Hitlan, this analytical opportunity would have been lost
forever.

Do these errors make Patrick Ip, Virginia Wong, Marco Ho, Joseph Lee and
Wilfred Wong of the University of Hong Kong junk scientists? Certainly
not. They are without doubt well-educated, well-meaning and smart
scientists. But in this case, they did bad work, bad on many levels:
testing a hypothesis that no one ever offered; drawing inappropriate
conclusions from their sloppy design, making calculation errors that the
initial peer reviewers missed; and failing to notice a surprising
relationship that should have been obvious to anyone with knowledge of
the poor excretor hypothesis. DeSoto and Hitlan deserve praise for
confronting the errors of the Hong Kong team, who should be held
accountable for doing bad work. Confronting error is a difficult and
risky exercise, especially when you're on the unpopular side of a
controversy. But in their conclusion, DeSoto and Hitlan state with great
eloquence why this is so important.

"Of utmost importance (which outweighs the discomfort of writing about
an error made by colleagues whom we know are generally competent
researchers) is that potential researchers who are trying to understand
what is and is not behind the rise in autism are not misled by even the
slightest misinformation."

Amen to that.

Here's a LINK <http://jcn.sagepub.com/cgi/content/abstract/22/11/1308>
to their report.

in Mark Blaxill <http://www.ageofautism.com/mark_blaxill/index.html>,
Science <http://www.ageofautism.com/science/index.html> | Permalink
<http://www.ageofautism.com/2007/11/when-smart-scie.html>

Digg This
<http://www.digg.com/submit?url=http://www.ageofautism.com/2007/11/when-smart-scie.html&phase=2>
| Save to del.icio.us <http://del.icio.us/post>


TrackBack

TrackBack URL for this entry:
http://www.typepad.com/t/trackback/2465202/23475908

Listed below are links to weblogs that reference WHEN SMART SCIENTISTS
MAKE STUPID MISTAKES
<http://www.ageofautism.com/2007/11/when-smart-scie.html>:


Reply all
Reply to author
Forward
0 new messages