Scientific Errors

[Amatista Sheeley]

Astronomersviewing supernova 1987A, pictured here, thought they saw a signal from a rapidly spinning neutron star too bizarre to comprehend. But the signal turned out to come from a quirk in the electronics of a camera used to aim the telescope.

A report in 2010 claimed that a weird form of life incorporates arsenic in place of phosphorus in biological molecules. This one sounded rather suspicious, but the evidence, at first glance, looked pretty good. Not so good at second glance, though. And arsenic-based life never made it into the textbooks.

In New Zealand, hedgehogs, like this one in Auckland, are invasive predators that hunt bird eggs, lizards and invertebrates. To protect native species, some conservationists support culling hedgehogs.

Science News was founded in 1921 as an independent, nonprofit source of accurate information on the latest news of science, medicine and technology. Today, our mission remains the same: to empower people to evaluate the news and the world around them. It is published by the Society for Science, a nonprofit 501(c)(3) membership organization dedicated to public engagement in scientific research and education (EIN 53-0196483).

Despite Walking with... being a documentary series, several paleontological inaccuracies appear throughout some of the shows. However, most of the errors are caused by newer and more recent discoveries. Here's a list of them.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

For many scientists, this is especially worrying in light of the reproducibility concerns. In 2005, epidemiologist John Ioannidis of Stanford University in California suggested that most published findings are false2; since then, a string of high-profile replication problems has forced scientists to rethink how they evaluate results.

The irony is that when UK statistician Ronald Fisher introduced the P value in the 1920s, he did not mean it to be a definitive test. He intended it simply as an informal way to judge whether evidence was significant in the old-fashioned sense: worthy of a second look. The idea was to run an experiment, then see if the results were consistent with what random chance might produce. Researchers would first set up a 'null hypothesis' that they wanted to disprove, such as there being no correlation or no difference between two groups. Next, they would play the devil's advocate and, assuming that this null hypothesis was in fact true, calculate the chances of getting results at least as extreme as what was actually observed. This probability was the P value. The smaller it was, suggested Fisher, the greater the likelihood that the straw-man null hypothesis was false.

For all the P value's apparent precision, Fisher intended it to be just one part of a fluid, non-numerical process that blended data and background knowledge to lead to scientific conclusions. But it soon got swept into a movement to make evidence-based decision-making as rigorous and objective as possible. This movement was spearheaded in the late 1920s by Fisher's bitter rivals, Polish mathematician Jerzy Neyman and UK statistician Egon Pearson, who introduced an alternative framework for data analysis that included statistical power, false positives, false negatives and many other concepts now familiar from introductory statistics classes. They pointedly left out the P value.

Statisticians have pointed to a number of measures that might help. To avoid the trap of thinking about results as significant or not significant, for example, Cumming thinks that researchers should always report effect sizes and confidence intervals. These convey what a P value does not: the magnitude and relative importance of an effect.

A related idea that is garnering attention is two-stage analysis, or 'preregistered replication', says political scientist and statistician Andrew Gelman of Columbia University in New York City. In this approach, exploratory and confirmatory analyses are approached differently and clearly labelled. Instead of doing four separate small studies and reporting the results in one paper, for instance, researchers would first do two small exploratory studies and gather potentially interesting findings without worrying too much about false alarms. Then, on the basis of these results, the authors would decide exactly how they planned to confirm the findings, and would publicly preregister their intentions in a database such as the Open Science Framework ( ). They would then conduct the replication studies and publish the results alongside those of the exploratory studies. This approach allows for freedom and flexibility in analyses, says Gelman, while providing enough rigour to reduce the number of false alarms being published.

Most scientists and engineers learn early that failing goes alongside any attempt to understand how things work and why. The more advanced and complex the science, the more likely that there will be many experiences of failure. Failure is a necessary and frequent feature of scientific research.

The problem is that we live in a society where failure is seen as always negative, something that reflects poorly on us. No one wants to have their scientific or engineering competence questioned and so researchers might avoid talking openly about failure. It can seem that only success is rewarded. Failure is often sanctioned.

We are not asking individual women researchers to be more open in disclosing failure. To do so in organizations that are flop-averse would not be good advice. Our long-term goal is to create greater opportunities in universities for STEM researchers to talk about failure as a routine aspect of scientific learning and discovery. Normalizing failure, creating more transparency about how important it is to fail and how to respond to failure is what I would encourage universities to integrate into organizational culture.

Q. Some women scientists say they must work harder than their male counterparts to be recognized as leaders and experts. Does that make it more difficult for women to publicly recount their failures?

A. Organizations that do failure well invest in building expertise in failure analysis in complex systems. Not all failures are the same and it is valuable to accumulate knowledge from different types of failure to make good decisions about next steps or solutions.

Recognizing someone who set out to do something important and failed means not blaming or shaming, focusing instead on what failed and why. This kind of approach developed for business may need to be adapted to fit university research settings, but it does suggest avenues for experimentation in approaching failure in new, generative ways.

In the process of reading the book and encountering some extraordinary claims about sleep, I decided to compare the facts it presented with the scientific literature. I found that the book consistently overstates the problem of lack of sleep, sometimes egregiously so. It misrepresents basic sleep research and contradicts its own sources.

The myths created by the book have spread in the popular culture and are being propagated by Walker and by other scientists in academic research. For example, in 2019, Walker published an academic paper that cited Why We Sleep 4 times just on its first page, meaning that he believes that the book abides by the academic, not the pop-science standards of accuracy (Section 14).

Note that the lowest mortality on the graph is at just below 7 hours and that mortality at 5 hours of sleep per night is basically the same if not lower than mortality at 8 hours of sleep.

Note: in this section, I only talk about acute sleep deprivation, i.e. being sleep deprived for one or several days. Chronic or externally imposed sleep deprivation is an entirely different matter and has no relation to sleep deprivation therapy.

SD is a rapid, safe, and effective therapy for depression. In recent years, this technique has passed the experimental developmental phase and reached the status of affordable clinical intervention for everyday clinical therapy of depressed patients with an increasing literature regarding its safety and efficacy.

[T]here is a very rare genetic disorder that starts with a progressive insomnia, emerging in midlife [fatal familial insomnia (FFI)]. Several months into the disease course, the patient stops sleeping altogether. By this stage, they have started to lose many basic brain and body functions. No drugs that we currently have will help the patient sleep. After twelve to eighteen months of no sleep, the patient will die. Though exceedingly rare, this disorder asserts that a lack of sleep can kill a human being.

It is reckless to claim that people with FFI die because of lack of sleep, given the amount of damage across the brain that accumulates in the course of the disease. Accordingly, FFI is considered a neurodegenerative disease. Looking at page 41 of the Encyclopedia of Sleep we discussed in the Interlude:

A disorder called fatal familial insomnia (FFI) is often presented as proof that sleep loss causes death in humans as it does in rats deprived by the forced walking method. However, FFI is a prion disease that affects all body organs and brain cells. There is little evidence that sleep induced by sedation can greatly extend life in FFI patients.

Later in the book (in Chapter 8) Walker provides the following graph that indicates that the average sleep time has decreased by more than 2 hours between 1940s and 2000s and that corroborates his claims of sleep loss epidemic, even if it was not declared by the WHO:

3a8082e126