Scientists leave out data to produce bogus findings

[pautrey2]

naturalnews.com printable article
Originally published January 5 2012


Busted! Scientists leave out data to produce bogus findings
by S. L. Baker, features writer

(NaturalNews) Clinical trials of drugs and other medical therapies are
carefully carried out and are the very gold standard of scientific
proof, right? According to an in-depth review of this question just
published in the British Medical Journal (BMJ,) the answer is no. In
fact, the BMJ is sounding the alarm that data reported by scientists
is too often not the truth -- because the researchers leave out
inconvenient evidence. The result of facts-gone-missing could well be
harming patients, spiking up healthcare costs by the selling of
medical treatments based on bogus findings, and threatening the very
integrity of medicine.

These warnings come from multiple papers released by the BMJ. The
whistle-blowing authors of these articles examined the extent, causes,
consequences of hidden facts, figures, and other data scientists
discover as they do human trials. It turns out this is no "once in a
while" kind of problem, either. The BMJ claims a "large proportion of
evidence from human trials is unreported, and much of what is reported
is done so inadequately."

In an editorial, Dr. Richard Lehman from the University of Oxford and
BMJ Clinical Epidemiology Editor, Dr. Elizabeth Loder, nail the
current state of medical research as a "culture of haphazard
publication and incomplete data disclosure." They call for full access
to raw trial data to allow better understanding of the benefits and
harms of many treatments.

Bottom line: when data is left out, the missing facts distort the
scientific record and published results of a study. This then leads
doctors to make potentially dangerous clinical decisions about what
drugs or procedures patients need because the docs are relying on
skewed and even bogus "evidence."

Conveniently missing facts left out of drug trials and more
Papers in the current issue of BMJ include a study by Dr. Beth Hart
and colleagues, which document how unpublished data is "conveniently
missing" from many published meta-analyses of drug trials. That's
right. Big Pharma's pills and potions are often pushed based on
studies that simply ignore and leave out major data about what was
really discovered about a medication lacking of benefits, potential
dangers, side effects and more. Dr. Hart's team argues that access to
full trial data is necessary to allow drugs to be independently
assessed.

Two additional studies show the requirements for mandatory trial
registration and timely sharing of results are poorly followed, if at
all. For example, it turns out that less than half of US National
Institutes of Health funded trials are published in a peer reviewed
journal within 30 months of completion and only 22 percent of trials
that are supposed to be subject to mandatory reporting had results
available within one year of completion.

"When the word mandatory turns out to mandate so little, the need for
stronger mechanisms of enforcement becomes very clear," the
researchers report.

And what happens when ethical, dedicated scientists try to assess true
harms vs benefits of Big Pharma drugs and other interventions? It's
not a pretty picture for their careers, apparently. Additional studies
published in the special BMJ issue highlight the many difficulties
these researchers face when they try to buck the system.

Dr. Lehman and Dr. Loder, however, are bravely speaking out and
directly saying that a concealment of data in clinical trials is
anything but unusual. They label this "a serious ethical breach" and
demand that clinical researchers who fail to disclose data "should be
subject to disciplinary action by professional organizations. These
changes have long been called for, and delay has already caused harm.
The evidence we publish shows that the current situation is a
disservice to research participants, patients, health systems, and the
whole endeavor of clinical medicine."

For more information:

http://www.bmj.com/

http://www.naturalnews.com/medical_errors.html




All content posted on this site is commentary or opinion and is
protected under Free Speech. Truth Publishing LLC takes sole
responsibility for all content. Truth Publishing sells no hard
products and earns no money from the recommendation of products.
NaturalNews.com is presented for educational and commentary purposes
only and should not be construed as professional advice from any
licensed practitioner. Truth Publishing assumes no responsibility for
the use or misuse of this material. For the full terms of usage of
this material, visit www.NaturalNews.com/terms.shtml




http://www.naturalnews.com/034577_quack_science_clinical_trials_BMJ.html

[Herman Rubin]

On 2012-01-09, pautrey2 <olivercr...@gmail.com> wrote:
> naturalnews.com printable article
> Originally published January 5 2012


> Busted! Scientists leave out data to produce bogus findings
> by S. L. Baker, features writer

> (NaturalNews) Clinical trials of drugs and other medical therapies are
> carefully carried out and are the very gold standard of scientific
> proof, right? According to an in-depth review of this question just
> published in the British Medical Journal (BMJ,) the answer is no. In
> fact, the BMJ is sounding the alarm that data reported by scientists
> is too often not the truth -- because the researchers leave out
> inconvenient evidence. The result of facts-gone-missing could well be
> harming patients, spiking up healthcare costs by the selling of
> medical treatments based on bogus findings, and threatening the very
> integrity of medicine.

The large set of your accusations are mainly false. But there
is something there, which I, in my professional capacity as a
theoretical statistician, have been aware of for decades, and
deplore as much as you do.

There is an essentially meaningless term called "statistical
significance". However, it is stressed in just about all
courses in statistics which are not at a mathematical level,
and even some of them. Typically, journal editors will not
publish a paper which does not have statistically significant
results. Thus, papers which do not produce positive results
do not get published, and as they do not, they do not get
submitted. This causes a number of significant, in the
practical sense, positive claims to be published, and also,
as the negative data is essentially unavailable, causes them
to seem very plausible.

As to what to do about it, the answer is unclear. The journals
are overloaded already, and could not publish all the negative
or inconclusive trials if they wanted to. There are various
reasons why competing researchers do not know of others' efforts.

In addition, the statistical methods are weak and cannot use all
the information available. Many of the statistical methods needed
are more than a half century old, and good theoretical statisticians
can produce new methods when appropriate. Not only physicians, but
also researchers in other fields, do not even see the basic
probability and conceptual statistics needed to formulate their
problems. Statistics is not a collection of rituals such that
one can, given a set of data, select one from the set and find
out what underlies the data. This is not possible. One cannot
extract statistical blood from a statistical turnip, and those
with poor backgrounds do not know what is possible, but just
use procedures as mantras.

I am personally willing to discuss this on a technical level,
and it needs that type of discussion.

--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hru...@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558

[pautrey2]

Friday, December 2, 2011 As of 12:00 AM
New York
50º|42º HEALTH
U.S. Edition Home
↓ More
Today's Paper
Video
Blogs
Journal Community
Subscribe
Log In

World » More
U.S. » More
New York » More
Business » More
Markets » More
Tech » More
Personal Finance » More
Life & Culture » More
Opinion » More
Careers » More
Real Estate » More
Small Business » More
Asia Europe Earnings Economy Health Law Autos Management Media
& Marketing Energy CFO Journal

More Industries

Accounting
Advertising
Airlines
Banking
Broadcasting & Entertainment
Chemicals
Computer Hardware
Computer Software
Consumer Products
Defense & Aerospace
Financial Services & Insurance
Food & Tobacco
Hospitality
Industrial Goods & Services
Marketing & Strategy
Media Agencies
Metals & Mining
Paper & Forest Products
Pharmaceutical & Biotech
Real Estate
Retail
Semiconductors
Telecommunications
Transportation
Utilities
TOP STORIES IN
Business
1 of 12

Prices Drop Amid Glut in Natural Gas
2 of 12

Coke: Fungicide Found in Orange Juice
3 of 12

Power Plants Top EPA List On Emissions
4 of 12

Banker Pay Heads to Ballot
5 of 12

Carlyle's 3 Founders Share Big Payday
6 of 12
PLAY
Oakland Stores Go From 'Pop-Up' to Permanent
7 of 12
Sears Suffers Setback As Large Lender Balks
8 of 12
Two Top Shareholders Dump Stakes In Diamond Foods
9 of 12
PLAY
Innovation Lives at U.S. Manufacturers
10 of 12
S&P: China Bank Loss Deferral a Step Back
11 of 12

News Corp.'s Fox Joins Battle to Buy Dodgers
12 of 12
PLAY
Stocks Outrun Tepid Economic Data
ADVERTISEMENT
prevnext
HEALTH INDUSTRYDECEMBER 2, 2011
Scientists' Elusive Goal: Reproducing Study Results
Article
Video
Stock Quotes
Comments (74)
MORE IN HEALTH »
Email
Print
Save
↓ More

smaller
Larger

By GAUTAM NAIK

Two years ago, a group of Boston researchers published a study
describing how they had destroyed cancer tumors by targeting a protein
called STK33. Scientists at biotechnology firm Amgen Inc. quickly
pounced on the idea and assigned two dozen researchers to try to
repeat the experiment with a goal of turning the findings into a drug.


WSJ's Gautam Naik has details of challenges scientists face in
reproducing claims made by medical journals. Photo: Sandy Huffaker/The
New York Times

It proved to be a waste of time and money. After six months of
intensive lab work, Amgen found it couldn't replicate the results and
scrapped the project.

"I was disappointed but not surprised," says Glenn Begley, vice
president of research at Amgen of Thousand Oaks, Calif. "More often
than not, we are unable to reproduce findings" published by
researchers in journals.

This is one of medicine's dirty secrets: Most results, including those
that appear in top-flight peer-reviewed journals, can't be reproduced.

Enlarge Image

Bayer
Researchers at Bayer's labs often find their experiments fail to match
claims made in the scientific literature.

"It's a very serious and disturbing issue because it obviously
misleads people" who implicitly trust findings published in a
respected peer-reviewed journal, says Bruce Alberts, editor of
Science. On Friday, the U.S. journal is devoting a large chunk of its
Dec. 2 issue to the problem of scientific replication.

Reproducibility is the foundation of all modern research, the standard
by which scientific claims are evaluated. In the U.S. alone,
biomedical research is a $100-billion-year enterprise. So when
published medical findings can't be validated by others, there are
major consequences.

Drug manufacturers rely heavily on early-stage academic research and
can waste millions of dollars on products if the original results are
later shown to be unreliable. Patients may enroll in clinical trials
based on conflicting data, and sometimes see no benefits or suffer
harmful side effects.

There is also a more insidious and pervasive problem: a preference for
positive results.


Unlike pharmaceutical companies, academic researchers rarely conduct
experiments in a "blinded" manner. This makes it easier to cherry-pick
statistical findings that support a positive result. In the quest for
jobs and funding, especially in an era of economic malaise, the
growing army of scientists need more successful experiments to their
name, not failed ones. An explosion of scientific and academic
journals has added to the pressure.

When it comes to results that can't be replicated, Dr. Alberts says
the increasing intricacy of experiments may be largely to blame. "It
has to do with the complexity of biology and the fact that methods
[used in labs] are getting more sophisticated," he says.

It is hard to assess whether the reproducibility problem has been
getting worse over the years; there are some signs suggesting it could
be. For example, the success rate of Phase 2 human trials—where a
drug's efficacy is measured—fell to 18% in 2008-2010 from 28% in
2006-2007, according to a global analysis published in the journal
Nature Reviews in May.

"Lack of reproducibility is one element in the decline in Phase 2
success," says Khusru Asadullah, a Bayer AG research executive.

In September, Bayer published a study describing how it had halted
nearly two-thirds of its early drug target projects because in-house
experiments failed to match claims made in the literature.

The German pharmaceutical company says that none of the claims it
attempted to validate were in papers that had been retracted or were
suspected of being flawed. Yet, even the data in the most prestigious
journals couldn't be confirmed, Bayer said.

Enlarge Image

In 2008, Pfizer Inc. made a high-profile bet, potentially worth more
than $725 million, that it could turn a 25-year-old Russian cold
medicine into an effective drug for Alzheimer's disease.

The idea was promising. Published by the journal Lancet, data from
researchers at Baylor College of Medicine and elsewhere suggested that
the drug, an antihistamine called Dimebon, could improve symptoms in
Alzheimer's patients. Later findings, presented by researchers at the
University of California Los Angeles at a Chicago conference, showed
that the drug appeared to prevent symptoms from worsening for up to 18
months.

"Statistically, the studies were very robust," says David Hung, chief
executive officer of Medivation Inc., a San Francisco biotech firm
that sponsored both studies.

In 2010, Medivation along with Pfizer released data from their own
clinical trial for Dimebon, involving nearly 600 patients with mild to
moderate Alzheimer's disease symptoms. The companies said they were
unable to reproduce the Lancet results. They also indicated they had
found no statistically significant difference between patients on the
drug versus the inactive placebo.

Pfizer and Medivation have just completed a one-year study of Dimebon
in over 1,000 patients, another effort to see if the drug could be a
potential treatment for Alzheimer's. They expect to announce the
results in coming months.

Scientists offer a few theories as to why duplicative results may be
so elusive. Two different labs can use slightly different equipment or
materials, leading to divergent results. The more variables there are
in an experiment, the more likely it is that small, unintended errors
will pile up and swing a lab's conclusions one way or the other. And,
of course, data that have been rigged, invented or fraudulently
altered won't stand up to future scrutiny.

According to a report published by the U.K.'s Royal Society, there
were 7.1 million researchers working globally across all scientific
fields—academic and corporate—in 2007, a 25% increase from five years
earlier.

From the Archives

Mistakes in Scientific Studies Surge 8/10/2011
"Among the more obvious yet unquantifiable reasons, there is immense
competition among laboratories and a pressure to publish," wrote Dr.
Asadullah and others from Bayer, in their September paper. "There is
also a bias toward publishing positive results, as it is easier to get
positive results accepted in good journals."

Science publications are under pressure, too. The number of research
journals has jumped 23% between 2001 and 2010, according to Elsevier,
which has analyzed the data. Their proliferation has ratcheted up
competitive pressure on even elite journals, which can generate buzz
by publishing splashy papers, typically containing positive findings,
to meet the demands of a 24-hour news cycle.

Dr. Alberts of Science acknowledges that journals increasingly have to
strike a balance between publishing studies "with broad appeal," while
making sure they aren't hyped.

Drugmakers also have a penchant for positive results. A 2008 study
published in the journal PLoS Medicine by researchers at the
University of California San Francisco looked at data from 33 new drug
applications submitted between 2001 and 2002 to the U.S. Food and Drug
Administration. The agency requires drug companies to provide all data
from clinical trials. However, the authors found that a quarter of the
trial data—most of it unfavorable—never got published because the
companies never submitted it to journals.

The upshot: doctors who end up prescribing the FDA-approved drugs
often don't get to see the unfavorable data.

"I would say that selectively publishing data is unethical because
there are human subjects involved," says Lisa Bero of UCSF and co-
author of the PLoS Medicine study.

In an email statement, a spokeswoman for the FDA said the agency
considers all data it is given when reviewing a drug but "does not
have the authority to control what a company chooses to publish."

Venture capital firms say they, too, are increasingly encountering
cases of nonrepeatable studies, and cite it as a key reason why they
are less willing to finance early-stage projects. Before investing in
very early-stage research, Atlas Ventures, a venture-capital firm that
backs biotech companies, now asks an outside lab to validate any
experimental data. In about half the cases the findings can't be
reproduced, says Bruce Booth, a partner in Atlas' Life Sciences group.

There have been several prominent cases of nonreproducibility in
recent months. For example, in September, the journal Science
partially retracted a 2009 paper linking a virus to chronic fatigue
syndrome because several labs couldn't replicate the published
results. The partial retraction came after two of the 13 study authors
went back to the blood samples they analyzed from chronic-fatigue
patients and found they were contaminated.

Some studies can't be redone for a more prosaic reason: the authors
won't make all their raw data available to rival scientists.

John Ioannidis of Stanford University recently attempted to reproduce
the findings of 18 papers published in the respected journal Nature
Genetics. He noted that 16 of these papers stated that the underlying
"gene expression" data for the studies were publicly available.

But the supplied data apparently weren't detailed enough, and results
from 16 of the 18 major papers couldn't fully be reproduced by Dr.
Ioannidis and his colleagues. "We have to take it [on faith] that the
findings are OK," said Dr. Ioannidis, an epidemiologist who studies
the credibility of medical research.

Veronique Kiermer, an editor at Nature, says she agrees with Dr.
Ioannidis' conclusions, noting that the findings have prompted the
journal to be more cautious when publishing large-scale genome
analyses.

When companies trying to find new drugs come up against the
nonreproducibility problem, the repercussions can be significant.

A few years ago, several groups of scientists began to seek out new
cancer drugs by targeting a protein called KRAS. The KRAS protein
transmits signals received on the outside of a cell to its interior
and is therefore crucial for regulating cell growth. But when certain
mutations occur, the signaling can become continuous. That triggers
excess growth such as tumors.

The mutated form of KRAS is believed to be responsible for more than
60% of pancreatic cancers and half of colorectal cancers. It has also
been implicated in the growth of tumors in many other organs, such as
the lung.

So scientists have been especially keen to impede KRAS and, thus, stop
the constant signaling that leads to tumor growth.

In 2008, researchers at Harvard Medical School used cell-culture
experiments to show that by inhibiting another protein, STK33, they
could prevent the growth of tumor cell lines driven by the
malfunctioning KRAS.

The finding galvanized researchers at Amgen, who first heard about the
experiments at a scientific conference. "Everyone was trying to do
this," recalls Dr. Begley of Amgen, which derives nearly half of its
revenues from cancer drugs and related treatments. "It was a really
big deal."

When the Harvard researchers published their results in the
prestigious journal Cell, in May 2009, Amgen moved swiftly to
capitalize on the findings.

At a meeting in the company's offices in Thousand Oaks, Calif., Dr.
Begley assigned a group of Amgen researchers the task of identifying
small molecules that might inhibit STK33. Another team got a more
basic job: reproduce the Harvard data.

"We're talking about hundreds of millions of dollars in downstream
investments" if the approach works," says Dr. Begley. "So we need to
be sure we're standing on something firm and solid."

But over the next few months, Dr. Begley and his team got increasingly
disheartened. Amgen scientists, it turned out, couldn't reproduce any
of the key findings published in Cell.

For example, there was no difference in the growth of cells where
STK33 was largely blocked, compared with a control group of cells
where STK33 wasn't blocked.

What could account for the irreproducibility of the results?

"In our opinion there were methodological issues" in Amgen's approach
that could have led to the different findings, says Claudia Scholl,
one of the lead authors of the original Cell paper.

Dr. Scholl points out, for example, that Amgen used a different
reagent to suppress STK33 than the one reported in Cell. Yet, she
acknowledges that even when slightly different reagents are used, "you
should be able to reproduce the results."

Now a cancer researcher at the University Hospital of Ulm in Germany,
Dr. Scholl says her team has reproduced the original Cell results
multiple times, and continues to have faith in STK33 as a cancer
target.

Amgen, however, killed its STK33 program. In September, two dozen of
the firm's scientists published a paper in the journal Cancer Research
describing their failure to reproduce the main Cell findings.

Dr. Begley suggests that academic scientists, like drug companies,
should perform more experiments in a "blinded" manner to reduce any
bias toward positive findings. Otherwise, he says, "there is a human
desire to get the results your boss wants you to get."

Adds Atlas' Mr. Booth: "Nobody gets a promotion from publishing a
negative study."

Write to Gautam Naik at gauta...@wsj.com


http://online.wsj.com/article/SB10001424052970203764804577059841672541590.html



On Jan 11, 11:47 am, Herman Rubin <hru...@skew.stat.purdue.edu> wrote:

[George Conklin]

"Herman Rubin" <hru...@skew.stat.purdue.edu> wrote in message
news:slrnjgrmad...@skew.stat.purdue.edu...

Yes, Herman, the problem is that negative findings are not published. In
fact, even the label "negative" suggests something bad is going on, even if
is a technical term. In the social sciences the reason why correlates of
income and social class always get published is that they usually are
positive findings, statistically significant. They show up all the time.
But the man aspects of life which show no class-specific correlations do not
show up in the jourhals, since they would be negative findings.