cauality with experimental data

[khacker]

I heard a guest lecture today on research methods and the expert argued that you cannot find causality in any other way other than doing experiments. This removes the possibility, if he is correct, for finding causality with survey data or statistical procedures like path analysis. I would like to know if this argument is generally accepted by statisticians. Thanks in advance for information on this subject.

Dr. Ken Hacker
Dept. of Communication Studies
New Mexico State University

[Rich Ulrich]

On Tue, 30 Apr 2013 13:30:00 -0700 (PDT), khacker
<kenneth...@gmail.com> wrote:

>I heard a guest lecture today on research methods and the expert
argued that you cannot find causality in any other way other than
doing experiments. This removes the possibility, if he is correct,
for finding causality with survey data or statistical procedures like
path analysis. I would like to know if this argument is generally
accepted by statisticians. Thanks in advance for information on this
subject.
>
>

He's wrong. So was Plato, at the other extreme, who
argued that you only know Truth by analyzing - in your
mind - the behavior and interaction of ideal forms.

Statisticians are often more aware of the weakness
of some particular *statistical* inference than are
folks who tend to take p-values on faith. But your
lecturer seems to have suggested that there should
be widespread nihilism, which is something that I'm not
aware of. I suspect that, overall, statisticians are like
everyone else, in being too willing to accept much that is
insufficiently demonstrated. We believe what we want
to believe.

Your statement about "doing experiments" would
seem to put a severe limit on, say, astro-physicists
who project rather detailed histories of the cosmos.
They do a *whole* lot extrapolation from a small number
of physical facts and formulas.

For comments with relevance to epidemiology and social
science, you can read the Wikip entry on "Bradford Hill
criteria." That is what led to the wide consensus on
tobacco, in the 1960s. Just about nothing is going to be
formally, widely accepted when based only on a single
survey or path analysis. - Everybody gets to argue for
their own omitted-factor or mitigating circumstance.

Earlier than Bradford Hill, Paul Meehl had written in 1955 about
relying on a "nomological net" with at least some reference
to inference.

--
Rich Ulrich

[khacker]

Thanks, Rich. I will look up the sources you cite. Good point about astronomy - where is the control group?! :) Ken

[Art Kendall]

I usually say that without manipulation of an independent variable
representing the causal construct and a single study is not _sufficient_
to conclude causality.
This is often stated (incompletely) as "Correlation does not prove
causation". Better would be "Observing correlation in a single study
does not prove causation by itself" or "Observing correlation in a
single study is insufficient to prove causality".

We need to show some form of correlation/association/tracking if we want
to argue for causality.

However in many fields it is impossible to manipulate an independent
variable. I cannot randomly assign countries to have nuclear weapons.
I cannot randomly force some people to be smokers and others to be
non-smokers. I cannot insert or remove planets from the solar system. etc.

Random assignment of a manipulated independent variable rules out many
rival hypotheses for explaining a relation/association.
Some writers have observed differences in achievement between socially
constructed groups called "races". However, currently psych has over
100 rival hypotheses that would have to be ruled out before one could
say the the differences were caused by race.

As Rich suggested Paul Meehl and Bradford Hill are good places to start.

An excellent place to read about rival hypotheses and quasi-experimental
work is
http://www.amazon.com/Experimental-Quasi-Experimental-Designs-Generalized-Inference/dp/0395615569

Abelson is an author who better formulates my view that statistics is
part of rhetoric. (Rhetoric in the technical meaning of the word.)A
shorter book but excellent book is.
http://www.amazon.com/Statistics-Principled-Argument-Robert-Abelson/dp/0805805281

It is also important to note that the inference from an experiment is to
the set of cases in the study. Whether one can generalize of perhaps
even project depends on other circumstances.

Another thing to keep in mind is that many phenomena can be caused by
more than one thing.

Art Kendall
Social Research Consultants

[khacker]

On Tuesday, April 30, 2013 2:30:00 PM UTC-6, khacker wrote:

This is very useful, Art. Thank you. While I agree about astronomy and other cases of scientific inquiry not being able to do experiments, much of social science inquiry does seem amenable to experimental data. For example, studies of Internet usage and political participation are assumed to be causal with correlational data but are not confirmed by experiments. Thanks much, Ken

[Art Kendall]

Quasi-experimental data can be useful in reasoning about causality, but
experimentation addresses causality directly. In many areas the
complete argument may have some points from experiments and other
points from quasi experimental studies.

It is interesting to note that after a psychologist received the Nobel
Prize in Economics, even economics which used to be almost exclusively
observational (quasiexperimental) is now adding much experimental work
in the mix.

Quasiexperimental studies can vary in how close they come to
experimental studies. The more of the usual plausible rival hypotheses
that can be ruled out, the stronger is the inference of causality.

Art Kendall
Social Research Consultants

[Rich Ulrich]

- The advantage of a good "experiment" is that it eliminates so
many competing explanations that otherwise must be dealt
with one by one.

I have not read the Abelson reference that Art gives, but I
have previously recommended his book, "Statistics as
Principled Argument." Whatever observations you have, they
ought to be described and connected by a convincing narrative.


If you want to delve into the deeper arguments of the
philosophy of science, one starting point is Thomas Kuhn's
book, "The Structure of Scientific Revolutions."

I spent even more time reading the well-edited procedings
of a super conference that discussed Kuhn -- "Criticism
and the Growth of Knowledge" (1970), edited by Lakatos.

--
Rich Ulrich

[Herman Rubin]

Determining causes can be a difficult problem. But often the
situation is the other way; dismissing causes because of lack
of correlation, or because of prejudice. One that I read long
ago was that temperature and sprouting seemed unrelated, but
actually has a positive effect, masked by the negative correlation
of temperature and rainfall. And as for race not having an effect
on whatever, probabilistic considerations show that this cannot be
the case, as racial mixing has not been that great; racial differences
have been found in DNA going back tens of thousands of years.

The null hypothesis of something not being a cause MAY be correct,
but statistical significance is not the way to test it, and for
a sufficiently large sample size will reject the null. Statistical
significance is wrong and needs to be replaced, but also one should
not be testing hypotheses, but deciding actions.


--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hru...@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558